2023-06-23T00:22:15.758 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-23T00:22:15.759 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} --archive /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882 --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882/orig.config.yaml 2023-06-23T00:22:15.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T00:22:15.907 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882 branch: wip-yuri4-testing-2023-06-21-0846-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} email: yweinste@redhat.com first_in_suite: false job_id: '7311882' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri4-testing-2023-06-21-0846-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2023-06-21-0846-quincy sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9037 sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri4-testing-2023-06-21-0846-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c targets: smithi055.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC75fSi2oSvKw3YNS36qaAND2urjwcW/v9YT2M3JHk9YXUiOrW0/APthQChdXlMU8j5NDTGTGQX6k2gD31Z7zcB7ZsZ0jxUAgZehlukP0Ss64qMcKhiMdbO/sbQeaHykNmMA1xQlfK91IILupQeAGwQihBbq8ZHEZEc/W9MppBndcjV0Wb/e1e69oAsVmjwd/supjKa6sIEQiCUSuD02zFhfdQIgb+j8GB4mGiQuSwBDT8bhOAtSnaQTXJCKWymD/FdqmSDIUA/dLRdRtxxFg2/u0ANS9350Y/0KWYcgcIPFvCdCAPtW5knw4XhZVnADAWnZ47cV/9MIiTJqYey0zp+t7W8K1Noxa3VkgUdnyKTcD7LTR4XHGFZ1dk4q3LYRV+4cJrjIVtIdC8+3YXaNBL2z0MRhEWtbIQ0vWtiJmOTgrfSKEiGw1blxYOgCiePEKjv956ggIEbQEeq4/DXdq0mJObpfpFc/jKwshNWscFv+gUuGfiEEJpp5kdu0IOdd98= smithi141.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIXpQjeegzj7I7Qy7Nhs98P522TO+rOlW/zrMBOYH1mMrcIPOJrEA61LILrT0RBuJUpdgtAZhwazYthRMQN5gohIHl4hQy/ahqG+m5nDAP2hZY5sGwad0HgEZB93bjdhPNRBLLKHHpRIIMXZImUsNNwdMpuQxeDtv2/Ts72Qkk/RZ5FpEUkGBIjJ5nfZVtFBF2Sjl1jrZ7eiuoKotuhY1UrO+wcR47RCAQ96xXChs2/h9+vZoag3hcqNGf5oSVZV4QpdWOGmlWPhPrdfkeitw9qVC0UnrKdwvYwuVoO9KdQQYT7JuQIHSei2vw18UeEqubajpjsf+/HePOhzDcNcO5LQc2UU4ouKrntIYH02/3TLUtIFLrvmNXrK6036O/Pg7mVFa9wZ11nelPmbeWCcHME3/8Kki/WyaRKYXZXw+uCCnG/OuCSliYYK49Ap1UvZuku3Qy3jObqADWbT0CMwfxeSX6enw3yPb4hbXmIoC3cd8tt9TMpakPr3x525uNkXc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - check-counter: counters: mgr: - min: 4 name: finisher-volumes.complete_latency.avgcount - expected_val: 0 name: finisher-volumes.queue_len - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestPerModuleFinsherThread teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 076bbebc42a14f7d568aaa78eabb0038327bcb23 timestamp: 2023-06-22_15:25:25 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-23T00:22:15.908 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa; will attempt to use it 2023-06-23T00:22:15.909 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks 2023-06-23T00:22:15.909 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-23T00:22:15.911 INFO:teuthology.task.internal:Checking packages... 2023-06-23T00:22:15.943 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '42ddc46e6e6582d8e29a5a982a81886fe028be2c' 2023-06-23T00:22:15.943 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-23T00:22:15.944 INFO:teuthology.packaging:ref: None 2023-06-23T00:22:15.944 INFO:teuthology.packaging:tag: None 2023-06-23T00:22:15.944 INFO:teuthology.packaging:branch: wip-yuri4-testing-2023-06-21-0846-quincy 2023-06-23T00:22:15.944 INFO:teuthology.packaging:sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T00:22:15.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri4-testing-2023-06-21-0846-quincy 2023-06-23T00:22:16.187 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-719.g42ddc46e 2023-06-23T00:22:16.187 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-23T00:22:16.195 INFO:teuthology.task.internal:no buildpackages task found 2023-06-23T00:22:16.195 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-23T00:22:16.213 INFO:teuthology.task.internal:Saving configuration 2023-06-23T00:22:16.240 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-23T00:22:16.247 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T00:22:16.277 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:17:06.968133', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJg8j10lPwIyrdbYxzeCZpuDwk6dWCNCV3XimBesJ7UuC1GTIx2eZN0jqqchuXYsjMeHXarAc5XZ7hMlhapBV5M='} 2023-06-23T00:22:16.303 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi141.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:17:06.966467', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMzwl3/+mdfc/xxSHjLIop3iKW4zeuwCC8sOyRN/lL+UFIcnRQh9ljB8fRSd06JmFEK6CPinu1sdCCDzDifWpjs='} 2023-06-23T00:22:16.304 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-23T00:22:16.311 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-06-23T00:22:16.312 INFO:teuthology.task.internal:roles: ubuntu@smithi141.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-06-23T00:22:16.312 INFO:teuthology.run_tasks:Running task console_log... 2023-06-23T00:22:16.398 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f41ce36c040>, signals=[15]) 2023-06-23T00:22:16.399 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-23T00:22:16.410 INFO:teuthology.task.internal:Opening connections... 2023-06-23T00:22:16.410 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:22:16.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:22:16.487 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:22:16.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:22:16.558 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-23T00:22:16.570 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2023-06-23T00:22:16.589 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2023-06-23T00:22:16.590 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2023-06-23T00:22:16.647 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T00:22:16.647 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8.4 (Ootpa)" 2023-06-23T00:22:16.647 INFO:teuthology.orchestra.run.smithi055.stdout:ID="rhel" 2023-06-23T00:22:16.648 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="fedora" 2023-06-23T00:22:16.648 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8.4" 2023-06-23T00:22:16.648 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8" 2023-06-23T00:22:16.649 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-23T00:22:16.649 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2023-06-23T00:22:16.649 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-23T00:22:16.649 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T00:22:16.650 INFO:teuthology.orchestra.run.smithi055.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T00:22:16.650 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T00:22:16.650 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:22:16.651 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T00:22:16.651 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-23T00:22:16.651 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T00:22:16.651 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-23T00:22:16.652 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2023-06-23T00:22:16.678 DEBUG:teuthology.orchestra.run.smithi141:> uname -m 2023-06-23T00:22:16.696 INFO:teuthology.orchestra.run.smithi141.stdout:x86_64 2023-06-23T00:22:16.697 DEBUG:teuthology.orchestra.run.smithi141:> cat /etc/os-release 2023-06-23T00:22:16.752 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T00:22:16.753 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="8.4 (Ootpa)" 2023-06-23T00:22:16.753 INFO:teuthology.orchestra.run.smithi141.stdout:ID="rhel" 2023-06-23T00:22:16.753 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="fedora" 2023-06-23T00:22:16.753 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="8.4" 2023-06-23T00:22:16.753 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el8" 2023-06-23T00:22:16.753 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-23T00:22:16.754 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31" 2023-06-23T00:22:16.754 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-23T00:22:16.754 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T00:22:16.754 INFO:teuthology.orchestra.run.smithi141.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T00:22:16.754 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T00:22:16.754 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:22:16.754 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T00:22:16.755 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-23T00:22:16.755 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T00:22:16.755 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-23T00:22:16.755 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2023-06-23T00:22:16.778 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-23T00:22:16.794 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-23T00:22:16.802 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-23T00:22:16.803 DEBUG:teuthology.orchestra.run.smithi055:> test '!' -e /home/ubuntu/cephtest 2023-06-23T00:22:16.807 DEBUG:teuthology.orchestra.run.smithi141:> test '!' -e /home/ubuntu/cephtest 2023-06-23T00:22:16.824 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-23T00:22:16.832 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-23T00:22:16.834 DEBUG:teuthology.orchestra.run.smithi055:> test -z $(ls -A /var/lib/ceph) 2023-06-23T00:22:16.863 DEBUG:teuthology.orchestra.run.smithi141:> test -z $(ls -A /var/lib/ceph) 2023-06-23T00:22:16.878 INFO:teuthology.orchestra.run.smithi055.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-23T00:22:16.880 INFO:teuthology.orchestra.run.smithi141.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-23T00:22:16.882 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-23T00:22:16.956 INFO:teuthology.run_tasks:Running task kernel... 2023-06-23T00:22:16.986 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-06-23T00:22:16.987 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-06-23T00:22:16.988 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-06-23T00:22:16.988 DEBUG:teuthology.orchestra.run.smithi055:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T00:22:16.989 DEBUG:teuthology.orchestra.run.smithi141:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T00:22:17.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:22:17.006 DEBUG:teuthology.orchestra.run.smithi141:> uname -r 2023-06-23T00:22:17.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:22:17.007 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2023-06-23T00:22:17.063 INFO:teuthology.orchestra.run.smithi141.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-06-23T00:22:17.063 INFO:teuthology.task.kernel:Running kernel on smithi141: 4.18.0-305.25.1.el8_4.x86_64 2023-06-23T00:22:17.064 DEBUG:teuthology.orchestra.run.smithi141:> sudo yum install -y kernel 2023-06-23T00:22:17.065 INFO:teuthology.orchestra.run.smithi055.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-06-23T00:22:17.065 INFO:teuthology.task.kernel:Running kernel on smithi055: 4.18.0-305.25.1.el8_4.x86_64 2023-06-23T00:22:17.066 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum install -y kernel 2023-06-23T00:22:18.415 INFO:teuthology.orchestra.run.smithi141.stdout:Updating Subscription Management repositories. 2023-06-23T00:22:18.416 INFO:teuthology.orchestra.run.smithi141.stdout:Unable to read consumer identity 2023-06-23T00:22:18.609 INFO:teuthology.orchestra.run.smithi055.stdout:Updating Subscription Management repositories. 2023-06-23T00:22:18.610 INFO:teuthology.orchestra.run.smithi055.stdout:Unable to read consumer identity 2023-06-23T00:22:21.881 INFO:teuthology.orchestra.run.smithi141.stdout:Copr repo for python3-asyncssh owned by ceph 1.6 kB/s | 3.5 kB 00:02 2023-06-23T00:22:22.403 INFO:teuthology.orchestra.run.smithi055.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-23T00:22:24.224 INFO:teuthology.orchestra.run.smithi141.stdout:Extra Packages for Enterprise Linux 6.8 MB/s | 16 MB 00:02 2023-06-23T00:22:24.728 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 6.8 MB/s | 16 MB 00:02 2023-06-23T00:22:27.502 INFO:teuthology.orchestra.run.smithi141.stdout:lab-extras 1.7 MB/s | 24 kB 00:00 2023-06-23T00:22:28.120 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 4.7 MB/s | 24 kB 00:00 2023-06-23T00:22:28.203 INFO:teuthology.orchestra.run.smithi141.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-06-23T00:22:28.204 INFO:teuthology.orchestra.run.smithi141.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-06-23T00:22:28.204 INFO:teuthology.orchestra.run.smithi141.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-06-23T00:22:28.347 INFO:teuthology.orchestra.run.smithi141.stdout:Dependencies resolved. 2023-06-23T00:22:28.348 INFO:teuthology.orchestra.run.smithi141.stdout:Nothing to do. 2023-06-23T00:22:28.348 INFO:teuthology.orchestra.run.smithi141.stdout:Complete! 2023-06-23T00:22:28.468 DEBUG:teuthology.orchestra.run.smithi141:> echo no | sudo yum reinstall kernel || true 2023-06-23T00:22:28.811 INFO:teuthology.orchestra.run.smithi141.stdout:Updating Subscription Management repositories. 2023-06-23T00:22:28.811 INFO:teuthology.orchestra.run.smithi141.stdout:Unable to read consumer identity 2023-06-23T00:22:28.837 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-06-23T00:22:28.837 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-06-23T00:22:28.837 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-06-23T00:22:28.988 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-06-23T00:22:28.989 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-06-23T00:22:28.989 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-06-23T00:22:29.108 INFO:teuthology.orchestra.run.smithi141.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 12:22:27 AM UTC. 2023-06-23T00:22:29.128 DEBUG:teuthology.orchestra.run.smithi055:> echo no | sudo yum reinstall kernel || true 2023-06-23T00:22:29.248 INFO:teuthology.orchestra.run.smithi141.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-23T00:22:29.249 INFO:teuthology.orchestra.run.smithi141.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:22:29.249 INFO:teuthology.orchestra.run.smithi141.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:22:29.305 INFO:teuthology.orchestra.run.smithi141.stderr:Error: No packages marked for reinstall. 2023-06-23T00:22:29.357 DEBUG:teuthology.orchestra.run.smithi141:> sudo yum reinstall -y kernel || true 2023-06-23T00:22:29.493 INFO:teuthology.orchestra.run.smithi055.stdout:Updating Subscription Management repositories. 2023-06-23T00:22:29.513 INFO:teuthology.orchestra.run.smithi055.stdout:Unable to read consumer identity 2023-06-23T00:22:29.697 INFO:teuthology.orchestra.run.smithi141.stdout:Updating Subscription Management repositories. 2023-06-23T00:22:29.697 INFO:teuthology.orchestra.run.smithi141.stdout:Unable to read consumer identity 2023-06-23T00:22:29.797 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:00:01 ago on Fri 23 Jun 2023 12:22:28 AM UTC. 2023-06-23T00:22:29.941 INFO:teuthology.orchestra.run.smithi055.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-23T00:22:29.941 INFO:teuthology.orchestra.run.smithi055.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:22:29.941 INFO:teuthology.orchestra.run.smithi055.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:22:29.963 INFO:teuthology.orchestra.run.smithi055.stderr:Error: No packages marked for reinstall. 2023-06-23T00:22:29.977 INFO:teuthology.orchestra.run.smithi141.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 12:22:27 AM UTC. 2023-06-23T00:22:30.013 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum reinstall -y kernel || true 2023-06-23T00:22:30.103 INFO:teuthology.orchestra.run.smithi141.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-23T00:22:30.103 INFO:teuthology.orchestra.run.smithi141.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:22:30.104 INFO:teuthology.orchestra.run.smithi141.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:22:30.123 INFO:teuthology.orchestra.run.smithi141.stderr:Error: No packages marked for reinstall. 2023-06-23T00:22:30.173 DEBUG:teuthology.orchestra.run.smithi141:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T00:22:30.301 INFO:teuthology.orchestra.run.smithi141.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:30.302 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:30.302 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:30.302 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-23T00:22:30.302 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-23T00:22:30.302 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-06-23T00:22:30.303 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi141.front.sepia.ceph.com, path=None, version=distro) 2023-06-23T00:22:30.303 DEBUG:teuthology.orchestra.run.smithi141:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T00:22:30.381 INFO:teuthology.orchestra.run.smithi055.stdout:Updating Subscription Management repositories. 2023-06-23T00:22:30.382 INFO:teuthology.orchestra.run.smithi055.stdout:Unable to read consumer identity 2023-06-23T00:22:30.428 INFO:teuthology.orchestra.run.smithi141.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:30.429 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:30.429 DEBUG:teuthology.orchestra.run.smithi141:> sudo rpm -qi grub2-tools 2023-06-23T00:22:30.456 INFO:teuthology.orchestra.run.smithi141.stdout:Name : grub2-tools 2023-06-23T00:22:30.456 INFO:teuthology.orchestra.run.smithi141.stdout:Epoch : 1 2023-06-23T00:22:30.456 INFO:teuthology.orchestra.run.smithi141.stdout:Version : 2.02 2023-06-23T00:22:30.457 INFO:teuthology.orchestra.run.smithi141.stdout:Release : 99.el8_4.1 2023-06-23T00:22:30.457 INFO:teuthology.orchestra.run.smithi141.stdout:Architecture: x86_64 2023-06-23T00:22:30.457 INFO:teuthology.orchestra.run.smithi141.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-06-23T00:22:30.458 INFO:teuthology.orchestra.run.smithi141.stdout:Group : System Environment/Base 2023-06-23T00:22:30.459 INFO:teuthology.orchestra.run.smithi141.stdout:Size : 9264631 2023-06-23T00:22:30.459 INFO:teuthology.orchestra.run.smithi141.stdout:License : GPLv3+ 2023-06-23T00:22:30.459 INFO:teuthology.orchestra.run.smithi141.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-06-23T00:22:30.459 INFO:teuthology.orchestra.run.smithi141.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-06-23T00:22:30.460 INFO:teuthology.orchestra.run.smithi141.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-06-23T00:22:30.460 INFO:teuthology.orchestra.run.smithi141.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-06-23T00:22:30.460 INFO:teuthology.orchestra.run.smithi141.stdout:Relocations : (not relocatable) 2023-06-23T00:22:30.460 INFO:teuthology.orchestra.run.smithi141.stdout:Packager : Red Hat, Inc. 2023-06-23T00:22:30.460 INFO:teuthology.orchestra.run.smithi141.stdout:Vendor : Red Hat, Inc. 2023-06-23T00:22:30.460 INFO:teuthology.orchestra.run.smithi141.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-23T00:22:30.460 INFO:teuthology.orchestra.run.smithi141.stdout:Summary : Support tools for GRUB. 2023-06-23T00:22:30.460 INFO:teuthology.orchestra.run.smithi141.stdout:Description : 2023-06-23T00:22:30.461 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:22:30.461 INFO:teuthology.orchestra.run.smithi141.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-23T00:22:30.461 INFO:teuthology.orchestra.run.smithi141.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-23T00:22:30.461 INFO:teuthology.orchestra.run.smithi141.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-23T00:22:30.461 INFO:teuthology.orchestra.run.smithi141.stdout:hardware devices. 2023-06-23T00:22:30.461 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:22:30.461 INFO:teuthology.orchestra.run.smithi141.stdout:This subpackage provides tools for support of all platforms. 2023-06-23T00:22:30.462 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-23T00:22:30.463 INFO:teuthology.task.kernel:Updating grub on smithi141 to boot 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:30.463 DEBUG:teuthology.orchestra.run.smithi141:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-23T00:22:30.680 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 12:22:28 AM UTC. 2023-06-23T00:22:30.824 INFO:teuthology.orchestra.run.smithi055.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-23T00:22:30.824 INFO:teuthology.orchestra.run.smithi055.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:22:30.824 INFO:teuthology.orchestra.run.smithi055.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:22:30.832 INFO:teuthology.orchestra.run.smithi055.stderr:Error: No packages marked for reinstall. 2023-06-23T00:22:30.882 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T00:22:31.014 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:31.014 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:31.015 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:31.015 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-23T00:22:31.015 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-23T00:22:31.015 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-06-23T00:22:31.015 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi055.front.sepia.ceph.com, path=None, version=distro) 2023-06-23T00:22:31.015 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T00:22:31.146 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:31.146 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:31.147 DEBUG:teuthology.orchestra.run.smithi055:> sudo rpm -qi grub2-tools 2023-06-23T00:22:31.175 INFO:teuthology.orchestra.run.smithi055.stdout:Name : grub2-tools 2023-06-23T00:22:31.176 INFO:teuthology.orchestra.run.smithi055.stdout:Epoch : 1 2023-06-23T00:22:31.176 INFO:teuthology.orchestra.run.smithi055.stdout:Version : 2.02 2023-06-23T00:22:31.176 INFO:teuthology.orchestra.run.smithi055.stdout:Release : 99.el8_4.1 2023-06-23T00:22:31.176 INFO:teuthology.orchestra.run.smithi055.stdout:Architecture: x86_64 2023-06-23T00:22:31.176 INFO:teuthology.orchestra.run.smithi055.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-06-23T00:22:31.176 INFO:teuthology.orchestra.run.smithi055.stdout:Group : System Environment/Base 2023-06-23T00:22:31.177 INFO:teuthology.orchestra.run.smithi055.stdout:Size : 9264631 2023-06-23T00:22:31.177 INFO:teuthology.orchestra.run.smithi055.stdout:License : GPLv3+ 2023-06-23T00:22:31.177 INFO:teuthology.orchestra.run.smithi055.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-06-23T00:22:31.177 INFO:teuthology.orchestra.run.smithi055.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-06-23T00:22:31.177 INFO:teuthology.orchestra.run.smithi055.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-06-23T00:22:31.177 INFO:teuthology.orchestra.run.smithi055.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-06-23T00:22:31.177 INFO:teuthology.orchestra.run.smithi055.stdout:Relocations : (not relocatable) 2023-06-23T00:22:31.178 INFO:teuthology.orchestra.run.smithi055.stdout:Packager : Red Hat, Inc. 2023-06-23T00:22:31.178 INFO:teuthology.orchestra.run.smithi055.stdout:Vendor : Red Hat, Inc. 2023-06-23T00:22:31.178 INFO:teuthology.orchestra.run.smithi055.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-23T00:22:31.178 INFO:teuthology.orchestra.run.smithi055.stdout:Summary : Support tools for GRUB. 2023-06-23T00:22:31.178 INFO:teuthology.orchestra.run.smithi055.stdout:Description : 2023-06-23T00:22:31.178 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:22:31.178 INFO:teuthology.orchestra.run.smithi055.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-23T00:22:31.179 INFO:teuthology.orchestra.run.smithi055.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-23T00:22:31.179 INFO:teuthology.orchestra.run.smithi055.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-23T00:22:31.179 INFO:teuthology.orchestra.run.smithi055.stdout:hardware devices. 2023-06-23T00:22:31.179 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:22:31.179 INFO:teuthology.orchestra.run.smithi055.stdout:This subpackage provides tools for support of all platforms. 2023-06-23T00:22:31.180 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-23T00:22:31.180 INFO:teuthology.task.kernel:Updating grub on smithi055 to boot 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:31.181 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-23T00:22:31.508 INFO:teuthology.orchestra.run.smithi141.stderr:Generating grub configuration file ... 2023-06-23T00:22:32.519 INFO:teuthology.orchestra.run.smithi055.stderr:Generating grub configuration file ... 2023-06-23T00:22:33.376 INFO:teuthology.orchestra.run.smithi141.stderr:done 2023-06-23T00:22:33.380 DEBUG:teuthology.orchestra.run.smithi141:> mktemp 2023-06-23T00:22:33.396 INFO:teuthology.orchestra.run.smithi141.stdout:/tmp/tmp.i3p4UQm3WA 2023-06-23T00:22:33.397 DEBUG:teuthology.orchestra.run.smithi141:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.i3p4UQm3WA 2023-06-23T00:22:33.466 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod 0666 /tmp/tmp.i3p4UQm3WA 2023-06-23T00:22:33.636 DEBUG:teuthology.orchestra.remote:smithi141:/tmp/tmp.i3p4UQm3WA is 6KB 2023-06-23T00:22:33.685 DEBUG:teuthology.orchestra.run.smithi141:> rm -fr /tmp/tmp.i3p4UQm3WA 2023-06-23T00:22:33.699 DEBUG:teuthology.orchestra.run.smithi141:> sudo /bin/ls /boot/loader/entries || true 2023-06-23T00:22:33.769 INFO:teuthology.orchestra.run.smithi141.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-06-23T00:22:33.769 INFO:teuthology.orchestra.run.smithi141.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-06-23T00:22:33.770 INFO:teuthology.orchestra.run.smithi141.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-06-23T00:22:33.770 INFO:teuthology.orchestra.run.smithi141.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-06-23T00:22:33.771 DEBUG:teuthology.orchestra.run.smithi141:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:33.918 DEBUG:teuthology.orchestra.run.smithi141:> sudo shutdown -r now 2023-06-23T00:22:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:done 2023-06-23T00:22:34.379 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2023-06-23T00:22:34.394 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.IZO2iRVhby 2023-06-23T00:22:34.394 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.IZO2iRVhby 2023-06-23T00:22:34.483 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.IZO2iRVhby 2023-06-23T00:22:34.599 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.IZO2iRVhby is 6KB 2023-06-23T00:22:34.650 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.IZO2iRVhby 2023-06-23T00:22:34.665 DEBUG:teuthology.orchestra.run.smithi055:> sudo /bin/ls /boot/loader/entries || true 2023-06-23T00:22:34.735 INFO:teuthology.orchestra.run.smithi055.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-06-23T00:22:34.736 INFO:teuthology.orchestra.run.smithi055.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-06-23T00:22:34.736 INFO:teuthology.orchestra.run.smithi055.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-06-23T00:22:34.736 INFO:teuthology.orchestra.run.smithi055.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-06-23T00:22:34.737 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:22:34.875 DEBUG:teuthology.orchestra.run.smithi055:> sudo shutdown -r now 2023-06-23T00:23:03.948 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-06-23T00:23:03.949 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-06-23T00:23:03.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:23:04.880 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-06-23T00:23:04.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-06-23T00:23:04.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:23:22.480 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-06-23T00:23:23.248 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-06-23T00:23:52.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-06-23T00:23:52.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:23:53.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-06-23T00:23:53.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:24:08.921 DEBUG:teuthology.orchestra.run.smithi055:> true 2023-06-23T00:24:09.248 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-06-23T00:24:09.248 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-06-23T00:24:09.249 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2023-06-23T00:24:09.303 INFO:teuthology.orchestra.run.smithi055.stdout:4.18.0-425.3.1.el8.x86_64 2023-06-23T00:24:09.303 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:24:09.304 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-23T00:24:09.304 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-06-23T00:24:10.305 DEBUG:teuthology.parallel:result is None 2023-06-23T00:24:10.864 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-06-23T00:24:40.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-06-23T00:24:40.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:24:41.250 DEBUG:teuthology.orchestra.run.smithi141:> true 2023-06-23T00:24:41.499 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-06-23T00:24:41.499 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-06-23T00:24:41.500 DEBUG:teuthology.orchestra.run.smithi141:> uname -r 2023-06-23T00:24:41.515 INFO:teuthology.orchestra.run.smithi141.stdout:4.18.0-425.3.1.el8.x86_64 2023-06-23T00:24:41.515 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:24:41.515 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-23T00:24:41.516 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-06-23T00:24:42.516 DEBUG:teuthology.parallel:result is None 2023-06-23T00:24:42.517 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-23T00:24:42.528 INFO:teuthology.task.internal:Creating test directory... 2023-06-23T00:24:42.528 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T00:24:42.533 DEBUG:teuthology.orchestra.run.smithi141:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T00:24:42.550 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-23T00:24:42.566 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-23T00:24:42.573 INFO:teuthology.task.internal:Creating archive directory... 2023-06-23T00:24:42.573 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T00:24:42.592 DEBUG:teuthology.orchestra.run.smithi141:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T00:24:42.631 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-23T00:24:42.644 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-23T00:24:42.645 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T00:24:42.661 DEBUG:teuthology.orchestra.run.smithi141:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T00:24:42.743 INFO:teuthology.orchestra.run.smithi141.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:24:42.763 INFO:teuthology.orchestra.run.smithi141.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:24:42.770 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:24:42.793 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:24:42.795 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-23T00:24:42.803 INFO:teuthology.task.internal:Configuring sudo... 2023-06-23T00:24:42.804 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T00:24:42.806 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T00:24:42.833 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-23T00:24:42.845 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-23T00:24:42.846 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T00:24:42.874 DEBUG:teuthology.orchestra.run.smithi141:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T00:24:42.889 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:24:42.975 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:24:43.038 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:24:43.085 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:24:43.122 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:24:43.122 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T00:24:43.188 DEBUG:teuthology.orchestra.run.smithi141:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:24:43.234 DEBUG:teuthology.orchestra.run.smithi141:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:24:43.295 DEBUG:teuthology.orchestra.run.smithi141:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:24:43.339 DEBUG:teuthology.orchestra.run.smithi141:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:24:43.378 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:24:43.378 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T00:24:43.442 DEBUG:teuthology.orchestra.run.smithi055:> sudo service rsyslog restart 2023-06-23T00:24:43.445 DEBUG:teuthology.orchestra.run.smithi141:> sudo service rsyslog restart 2023-06-23T00:24:43.490 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T00:24:43.532 INFO:teuthology.orchestra.run.smithi141.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T00:24:43.837 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-23T00:24:43.858 INFO:teuthology.task.internal:Starting timer... 2023-06-23T00:24:43.858 INFO:teuthology.run_tasks:Running task pcp... 2023-06-23T00:24:43.868 INFO:teuthology.run_tasks:Running task selinux... 2023-06-23T00:24:43.876 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-06-23T00:24:43.877 DEBUG:teuthology.orchestra.run.smithi055:> sudo service auditd rotate 2023-06-23T00:24:43.936 INFO:teuthology.orchestra.run.smithi055.stdout:Rotating logs: [ OK ] 2023-06-23T00:24:43.938 DEBUG:teuthology.orchestra.run.smithi141:> sudo service auditd rotate 2023-06-23T00:24:43.994 INFO:teuthology.orchestra.run.smithi141.stdout:Rotating logs: [ OK ] 2023-06-23T00:24:43.996 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-23T00:24:43.996 DEBUG:teuthology.orchestra.run.smithi055:> /usr/sbin/getenforce 2023-06-23T00:24:44.026 INFO:teuthology.orchestra.run.smithi055.stdout:Permissive 2023-06-23T00:24:44.027 DEBUG:teuthology.orchestra.run.smithi141:> /usr/sbin/getenforce 2023-06-23T00:24:44.049 INFO:teuthology.orchestra.run.smithi141.stdout:Permissive 2023-06-23T00:24:44.050 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi055.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi141.front.sepia.ceph.com': 'permissive'} 2023-06-23T00:24:44.050 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T00:24:44.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:24:44.093 DEBUG:teuthology.orchestra.run.smithi141:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T00:24:44.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:24:44.119 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-23T00:24:44.119 DEBUG:teuthology.orchestra.run.smithi055:> sudo /usr/sbin/setenforce permissive 2023-06-23T00:24:44.165 DEBUG:teuthology.orchestra.run.smithi141:> sudo /usr/sbin/setenforce permissive 2023-06-23T00:24:44.192 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-23T00:24:44.203 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-06-23T00:24:44.204 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-23T00:24:44.228 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-23T00:24:44.230 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi055.front.sepia.ceph.com,smithi141.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-23T00:32:50.485 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi055.front.sepia.ceph.com'), Remote(name='ubuntu@smithi141.front.sepia.ceph.com')] 2023-06-23T00:32:50.487 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-06-23T00:32:50.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:32:50.558 DEBUG:teuthology.orchestra.run.smithi055:> true 2023-06-23T00:32:50.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-06-23T00:32:50.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:32:50.694 DEBUG:teuthology.orchestra.run.smithi141:> true 2023-06-23T00:32:50.759 INFO:teuthology.run_tasks:Running task clock... 2023-06-23T00:32:50.776 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-23T00:32:50.777 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T00:32:50.778 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T00:32:50.780 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T00:32:50.781 DEBUG:teuthology.orchestra.run.smithi141:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T00:32:50.810 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T00:32:50.825 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T00:32:50.828 INFO:teuthology.orchestra.run.smithi141.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T00:32:50.843 INFO:teuthology.orchestra.run.smithi141.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T00:32:50.862 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: ntpd: command not found 2023-06-23T00:32:50.873 INFO:teuthology.orchestra.run.smithi055.stdout:506 Cannot talk to daemon 2023-06-23T00:32:50.875 INFO:teuthology.orchestra.run.smithi141.stderr:sudo: ntpd: command not found 2023-06-23T00:32:50.886 INFO:teuthology.orchestra.run.smithi141.stdout:506 Cannot talk to daemon 2023-06-23T00:32:50.887 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T00:32:50.900 INFO:teuthology.orchestra.run.smithi141.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T00:32:50.901 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T00:32:50.913 INFO:teuthology.orchestra.run.smithi141.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T00:32:50.982 INFO:teuthology.orchestra.run.smithi141.stderr:bash: ntpq: command not found 2023-06-23T00:32:50.983 INFO:teuthology.orchestra.run.smithi055.stderr:bash: ntpq: command not found 2023-06-23T00:32:50.984 INFO:teuthology.orchestra.run.smithi141.stdout:210 Number of sources = 4 2023-06-23T00:32:50.984 INFO:teuthology.orchestra.run.smithi141.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T00:32:50.985 INFO:teuthology.orchestra.run.smithi141.stdout:=============================================================================== 2023-06-23T00:32:50.985 INFO:teuthology.orchestra.run.smithi141.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T00:32:50.985 INFO:teuthology.orchestra.run.smithi141.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T00:32:50.985 INFO:teuthology.orchestra.run.smithi141.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T00:32:50.985 INFO:teuthology.orchestra.run.smithi141.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T00:32:50.986 INFO:teuthology.orchestra.run.smithi055.stdout:210 Number of sources = 4 2023-06-23T00:32:50.987 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T00:32:50.987 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2023-06-23T00:32:50.987 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T00:32:50.987 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T00:32:50.987 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T00:32:50.987 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T00:32:50.988 INFO:teuthology.run_tasks:Running task install... 2023-06-23T00:32:50.998 DEBUG:teuthology.task.install:project ceph 2023-06-23T00:32:50.998 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}} 2023-06-23T00:32:50.998 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'} 2023-06-23T00:32:50.999 INFO:teuthology.task.install:Using flavor: default 2023-06-23T00:32:51.005 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-23T00:32:51.005 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-23T00:32:51.006 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-23T00:32:51.007 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T00:32:51.009 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-23T00:32:51.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T00:32:51.156 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-23T00:32:51.156 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-23T00:32:51.224 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-23T00:32:51.224 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-23T00:32:51.331 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T00:32:51.332 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:32:51.332 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T00:32:51.360 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-23T00:32:51.360 DEBUG:teuthology.orchestra.run.smithi141:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T00:32:51.425 DEBUG:teuthology.orchestra.run.smithi141:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T00:32:51.437 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T00:32:51.438 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:32:51.438 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T00:32:51.465 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-23T00:32:51.466 DEBUG:teuthology.orchestra.run.smithi055:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T00:32:51.500 DEBUG:teuthology.orchestra.run.smithi141:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T00:32:51.533 DEBUG:teuthology.orchestra.run.smithi055:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T00:32:51.564 INFO:teuthology.orchestra.run.smithi141.stdout:check_obsoletes = 1 2023-06-23T00:32:51.565 DEBUG:teuthology.orchestra.run.smithi141:> sudo yum clean all 2023-06-23T00:32:51.609 DEBUG:teuthology.orchestra.run.smithi055:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T00:32:51.673 INFO:teuthology.orchestra.run.smithi055.stdout:check_obsoletes = 1 2023-06-23T00:32:51.675 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum clean all 2023-06-23T00:32:51.954 INFO:teuthology.orchestra.run.smithi141.stdout:Updating Subscription Management repositories. 2023-06-23T00:32:52.085 INFO:teuthology.orchestra.run.smithi055.stdout:Updating Subscription Management repositories. 2023-06-23T00:32:52.379 INFO:teuthology.orchestra.run.smithi055.stdout:44 files removed 2023-06-23T00:32:52.383 INFO:teuthology.orchestra.run.smithi141.stdout:44 files removed 2023-06-23T00:32:52.432 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-23T00:32:52.434 DEBUG:teuthology.orchestra.run.smithi141:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-23T00:32:52.783 INFO:teuthology.orchestra.run.smithi141.stdout:Updating Subscription Management repositories. 2023-06-23T00:32:52.804 INFO:teuthology.orchestra.run.smithi055.stdout:Updating Subscription Management repositories. 2023-06-23T00:32:54.018 INFO:teuthology.orchestra.run.smithi141.stdout:ceph packages for x86_64 116 kB/s | 78 kB 00:00 2023-06-23T00:32:54.066 INFO:teuthology.orchestra.run.smithi055.stdout:ceph packages for x86_64 123 kB/s | 78 kB 00:00 2023-06-23T00:32:54.397 INFO:teuthology.orchestra.run.smithi141.stdout:ceph noarch packages 27 kB/s | 9.2 kB 00:00 2023-06-23T00:32:54.428 INFO:teuthology.orchestra.run.smithi055.stdout:ceph noarch packages 29 kB/s | 9.2 kB 00:00 2023-06-23T00:32:54.919 INFO:teuthology.orchestra.run.smithi141.stdout:ceph source packages 2.9 kB/s | 1.4 kB 00:00 2023-06-23T00:32:54.996 INFO:teuthology.orchestra.run.smithi055.stdout:ceph source packages 2.7 kB/s | 1.4 kB 00:00 2023-06-23T00:32:57.139 INFO:teuthology.orchestra.run.smithi141.stdout:Copr repo for python3-asyncssh owned by ceph 1.6 kB/s | 3.5 kB 00:02 2023-06-23T00:32:57.454 INFO:teuthology.orchestra.run.smithi055.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-23T00:32:58.227 INFO:teuthology.orchestra.run.smithi141.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-06-23T00:32:58.241 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-06-23T00:33:01.416 INFO:teuthology.orchestra.run.smithi141.stdout:lab-extras 4.9 MB/s | 24 kB 00:00 2023-06-23T00:33:01.799 INFO:teuthology.orchestra.run.smithi141.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 126 MB/s | 38 MB 00:00 2023-06-23T00:33:01.882 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-06-23T00:33:02.308 INFO:teuthology.orchestra.run.smithi055.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 112 MB/s | 38 MB 00:00 2023-06-23T00:33:07.346 INFO:teuthology.orchestra.run.smithi141.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 119 MB/s | 33 MB 00:00 2023-06-23T00:33:08.360 INFO:teuthology.orchestra.run.smithi055.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 105 MB/s | 33 MB 00:00 2023-06-23T00:33:15.183 INFO:teuthology.orchestra.run.smithi141.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T00:33:15.184 INFO:teuthology.orchestra.run.smithi141.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T00:33:15.194 INFO:teuthology.orchestra.run.smithi141.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-23T00:33:15.195 INFO:teuthology.orchestra.run.smithi141.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-23T00:33:15.196 INFO:teuthology.orchestra.run.smithi141.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-23T00:33:15.197 INFO:teuthology.orchestra.run.smithi141.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T00:33:15.198 INFO:teuthology.orchestra.run.smithi141.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T00:33:15.201 INFO:teuthology.orchestra.run.smithi141.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-23T00:33:15.202 INFO:teuthology.orchestra.run.smithi141.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-23T00:33:15.694 INFO:teuthology.orchestra.run.smithi141.stdout:Dependencies resolved. 2023-06-23T00:33:15.705 INFO:teuthology.orchestra.run.smithi141.stdout:====================================================================================================================== 2023-06-23T00:33:15.705 INFO:teuthology.orchestra.run.smithi141.stdout: Package Arch Version Repository Size 2023-06-23T00:33:15.706 INFO:teuthology.orchestra.run.smithi141.stdout:====================================================================================================================== 2023-06-23T00:33:15.706 INFO:teuthology.orchestra.run.smithi141.stdout:Installing: 2023-06-23T00:33:15.706 INFO:teuthology.orchestra.run.smithi141.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T00:33:15.706 INFO:teuthology.orchestra.run.smithi141.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-23T00:33:15.706 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-23T00:33:15.706 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-23T00:33:15.706 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-23T00:33:15.707 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-23T00:33:15.707 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-23T00:33:15.707 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-23T00:33:15.707 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-23T00:33:15.707 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-23T00:33:15.707 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T00:33:15.707 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-23T00:33:15.707 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-23T00:33:15.708 INFO:teuthology.orchestra.run.smithi141.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-23T00:33:15.708 INFO:teuthology.orchestra.run.smithi141.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-23T00:33:15.708 INFO:teuthology.orchestra.run.smithi141.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-23T00:33:15.708 INFO:teuthology.orchestra.run.smithi141.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-23T00:33:15.708 INFO:teuthology.orchestra.run.smithi141.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T00:33:15.708 INFO:teuthology.orchestra.run.smithi141.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T00:33:15.708 INFO:teuthology.orchestra.run.smithi141.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-23T00:33:15.709 INFO:teuthology.orchestra.run.smithi141.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-23T00:33:15.709 INFO:teuthology.orchestra.run.smithi141.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-23T00:33:15.709 INFO:teuthology.orchestra.run.smithi141.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T00:33:15.709 INFO:teuthology.orchestra.run.smithi141.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-23T00:33:15.709 INFO:teuthology.orchestra.run.smithi141.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T00:33:15.709 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-23T00:33:15.709 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-23T00:33:15.710 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-23T00:33:15.710 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-23T00:33:15.710 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-23T00:33:15.710 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T00:33:15.710 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-23T00:33:15.710 INFO:teuthology.orchestra.run.smithi141.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-23T00:33:15.710 INFO:teuthology.orchestra.run.smithi141.stdout:Upgrading: 2023-06-23T00:33:15.710 INFO:teuthology.orchestra.run.smithi141.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T00:33:15.711 INFO:teuthology.orchestra.run.smithi141.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-23T00:33:15.711 INFO:teuthology.orchestra.run.smithi141.stdout:Installing dependencies: 2023-06-23T00:33:15.711 INFO:teuthology.orchestra.run.smithi141.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T00:33:15.711 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-23T00:33:15.711 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-23T00:33:15.711 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-23T00:33:15.711 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-23T00:33:15.711 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-23T00:33:15.712 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T00:33:15.712 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-23T00:33:15.712 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-23T00:33:15.712 INFO:teuthology.orchestra.run.smithi141.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T00:33:15.712 INFO:teuthology.orchestra.run.smithi141.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T00:33:15.712 INFO:teuthology.orchestra.run.smithi141.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-23T00:33:15.712 INFO:teuthology.orchestra.run.smithi141.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T00:33:15.713 INFO:teuthology.orchestra.run.smithi141.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T00:33:15.713 INFO:teuthology.orchestra.run.smithi141.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-23T00:33:15.713 INFO:teuthology.orchestra.run.smithi141.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-23T00:33:15.713 INFO:teuthology.orchestra.run.smithi141.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-23T00:33:15.713 INFO:teuthology.orchestra.run.smithi141.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T00:33:15.713 INFO:teuthology.orchestra.run.smithi141.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-23T00:33:15.713 INFO:teuthology.orchestra.run.smithi141.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T00:33:15.714 INFO:teuthology.orchestra.run.smithi141.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-23T00:33:15.714 INFO:teuthology.orchestra.run.smithi141.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-23T00:33:15.714 INFO:teuthology.orchestra.run.smithi141.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-23T00:33:15.714 INFO:teuthology.orchestra.run.smithi141.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T00:33:15.714 INFO:teuthology.orchestra.run.smithi141.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-23T00:33:15.714 INFO:teuthology.orchestra.run.smithi141.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T00:33:15.714 INFO:teuthology.orchestra.run.smithi141.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T00:33:15.714 INFO:teuthology.orchestra.run.smithi141.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T00:33:15.715 INFO:teuthology.orchestra.run.smithi141.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T00:33:15.715 INFO:teuthology.orchestra.run.smithi141.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-23T00:33:15.715 INFO:teuthology.orchestra.run.smithi141.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T00:33:15.715 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T00:33:15.715 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T00:33:15.715 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T00:33:15.715 INFO:teuthology.orchestra.run.smithi141.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T00:33:15.716 INFO:teuthology.orchestra.run.smithi141.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T00:33:15.716 INFO:teuthology.orchestra.run.smithi141.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T00:33:15.716 INFO:teuthology.orchestra.run.smithi141.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T00:33:15.716 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T00:33:15.716 INFO:teuthology.orchestra.run.smithi141.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-23T00:33:15.716 INFO:teuthology.orchestra.run.smithi141.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-23T00:33:15.716 INFO:teuthology.orchestra.run.smithi141.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T00:33:15.717 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T00:33:15.717 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T00:33:15.717 INFO:teuthology.orchestra.run.smithi141.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T00:33:15.717 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T00:33:15.717 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T00:33:15.717 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T00:33:15.717 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T00:33:15.717 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T00:33:15.718 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T00:33:15.718 INFO:teuthology.orchestra.run.smithi141.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T00:33:15.718 INFO:teuthology.orchestra.run.smithi141.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T00:33:15.718 INFO:teuthology.orchestra.run.smithi141.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T00:33:15.718 INFO:teuthology.orchestra.run.smithi141.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T00:33:15.718 INFO:teuthology.orchestra.run.smithi141.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T00:33:15.718 INFO:teuthology.orchestra.run.smithi141.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T00:33:15.719 INFO:teuthology.orchestra.run.smithi141.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T00:33:15.719 INFO:teuthology.orchestra.run.smithi141.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T00:33:15.719 INFO:teuthology.orchestra.run.smithi141.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T00:33:15.719 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T00:33:15.719 INFO:teuthology.orchestra.run.smithi141.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T00:33:15.719 INFO:teuthology.orchestra.run.smithi141.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T00:33:15.719 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T00:33:15.719 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T00:33:15.720 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T00:33:15.720 INFO:teuthology.orchestra.run.smithi141.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T00:33:15.720 INFO:teuthology.orchestra.run.smithi141.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T00:33:15.720 INFO:teuthology.orchestra.run.smithi141.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T00:33:15.720 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T00:33:15.720 INFO:teuthology.orchestra.run.smithi141.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-23T00:33:15.720 INFO:teuthology.orchestra.run.smithi141.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T00:33:15.721 INFO:teuthology.orchestra.run.smithi141.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T00:33:15.721 INFO:teuthology.orchestra.run.smithi141.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T00:33:15.721 INFO:teuthology.orchestra.run.smithi141.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T00:33:15.721 INFO:teuthology.orchestra.run.smithi141.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T00:33:15.721 INFO:teuthology.orchestra.run.smithi141.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T00:33:15.721 INFO:teuthology.orchestra.run.smithi141.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T00:33:15.721 INFO:teuthology.orchestra.run.smithi141.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T00:33:15.722 INFO:teuthology.orchestra.run.smithi141.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T00:33:15.722 INFO:teuthology.orchestra.run.smithi141.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T00:33:15.722 INFO:teuthology.orchestra.run.smithi141.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-23T00:33:15.722 INFO:teuthology.orchestra.run.smithi141.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T00:33:15.722 INFO:teuthology.orchestra.run.smithi141.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T00:33:15.722 INFO:teuthology.orchestra.run.smithi141.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T00:33:15.722 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:33:15.722 INFO:teuthology.orchestra.run.smithi141.stdout:Transaction Summary 2023-06-23T00:33:15.723 INFO:teuthology.orchestra.run.smithi141.stdout:====================================================================================================================== 2023-06-23T00:33:15.723 INFO:teuthology.orchestra.run.smithi141.stdout:Install 118 Packages 2023-06-23T00:33:15.723 INFO:teuthology.orchestra.run.smithi141.stdout:Upgrade 2 Packages 2023-06-23T00:33:15.723 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:33:15.724 INFO:teuthology.orchestra.run.smithi141.stdout:Total download size: 201 M 2023-06-23T00:33:15.724 INFO:teuthology.orchestra.run.smithi141.stdout:Downloading Packages: 2023-06-23T00:33:16.136 INFO:teuthology.orchestra.run.smithi141.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 71 kB/s | 6.4 kB 00:00 2023-06-23T00:33:16.327 INFO:teuthology.orchestra.run.smithi141.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 4.5 MB/s | 873 kB 00:00 2023-06-23T00:33:16.351 INFO:teuthology.orchestra.run.smithi141.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 6.8 MB/s | 162 kB 00:00 2023-06-23T00:33:16.581 INFO:teuthology.orchestra.run.smithi141.stdout:(4/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 10 MB/s | 2.3 MB 00:00 2023-06-23T00:33:16.658 INFO:teuthology.orchestra.run.smithi141.stdout:(5/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 8.4 MB/s | 5.2 MB 00:00 2023-06-23T00:33:16.717 INFO:teuthology.orchestra.run.smithi141.stdout:(6/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 12 MB/s | 1.6 MB 00:00 2023-06-23T00:33:16.978 INFO:teuthology.orchestra.run.smithi141.stdout:(7/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 12 MB/s | 3.9 MB 00:00 2023-06-23T00:33:17.197 INFO:teuthology.orchestra.run.smithi055.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T00:33:17.198 INFO:teuthology.orchestra.run.smithi055.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T00:33:17.209 INFO:teuthology.orchestra.run.smithi055.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-23T00:33:17.210 INFO:teuthology.orchestra.run.smithi055.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-23T00:33:17.210 INFO:teuthology.orchestra.run.smithi055.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-23T00:33:17.211 INFO:teuthology.orchestra.run.smithi055.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T00:33:17.213 INFO:teuthology.orchestra.run.smithi055.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T00:33:17.215 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-23T00:33:17.216 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-23T00:33:17.730 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-06-23T00:33:17.742 INFO:teuthology.orchestra.run.smithi055.stdout:====================================================================================================================== 2023-06-23T00:33:17.743 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2023-06-23T00:33:17.743 INFO:teuthology.orchestra.run.smithi055.stdout:====================================================================================================================== 2023-06-23T00:33:17.743 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-06-23T00:33:17.743 INFO:teuthology.orchestra.run.smithi055.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T00:33:17.743 INFO:teuthology.orchestra.run.smithi055.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-23T00:33:17.743 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-23T00:33:17.744 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-23T00:33:17.744 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-23T00:33:17.744 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-23T00:33:17.744 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-23T00:33:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-23T00:33:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-23T00:33:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-23T00:33:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T00:33:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-23T00:33:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-23T00:33:17.745 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-23T00:33:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-23T00:33:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-23T00:33:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-23T00:33:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T00:33:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T00:33:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-23T00:33:17.746 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-23T00:33:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-23T00:33:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T00:33:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-23T00:33:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T00:33:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-23T00:33:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-23T00:33:17.747 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-23T00:33:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-23T00:33:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-23T00:33:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T00:33:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-23T00:33:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-23T00:33:17.748 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrading: 2023-06-23T00:33:17.748 INFO:teuthology.orchestra.run.smithi055.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T00:33:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-23T00:33:17.749 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-06-23T00:33:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T00:33:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-23T00:33:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-23T00:33:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-23T00:33:17.749 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-23T00:33:17.750 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-23T00:33:17.750 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T00:33:17.750 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-23T00:33:17.750 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-23T00:33:17.750 INFO:teuthology.orchestra.run.smithi055.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T00:33:17.750 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T00:33:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-23T00:33:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T00:33:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T00:33:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-23T00:33:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-23T00:33:17.751 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-23T00:33:17.752 INFO:teuthology.orchestra.run.smithi055.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T00:33:17.752 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-23T00:33:17.752 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T00:33:17.752 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-23T00:33:17.752 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-23T00:33:17.752 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-23T00:33:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T00:33:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-23T00:33:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T00:33:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T00:33:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T00:33:17.753 INFO:teuthology.orchestra.run.smithi055.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T00:33:17.754 INFO:teuthology.orchestra.run.smithi055.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-23T00:33:17.754 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T00:33:17.755 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T00:33:17.755 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T00:33:17.755 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T00:33:17.755 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T00:33:17.755 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T00:33:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T00:33:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T00:33:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T00:33:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-23T00:33:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-23T00:33:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T00:33:17.756 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T00:33:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T00:33:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T00:33:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T00:33:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T00:33:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T00:33:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T00:33:17.757 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T00:33:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T00:33:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T00:33:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T00:33:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T00:33:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T00:33:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T00:33:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T00:33:17.758 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T00:33:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T00:33:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T00:33:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T00:33:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T00:33:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T00:33:17.759 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T00:33:17.760 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T00:33:17.760 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T00:33:17.760 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T00:33:17.760 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T00:33:17.760 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T00:33:17.761 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T00:33:17.761 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-23T00:33:17.761 INFO:teuthology.orchestra.run.smithi055.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T00:33:17.761 INFO:teuthology.orchestra.run.smithi055.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T00:33:17.761 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T00:33:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T00:33:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T00:33:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T00:33:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T00:33:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T00:33:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T00:33:17.762 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T00:33:17.763 INFO:teuthology.orchestra.run.smithi055.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-23T00:33:17.763 INFO:teuthology.orchestra.run.smithi055.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T00:33:17.763 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T00:33:17.763 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T00:33:17.763 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:33:17.763 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-06-23T00:33:17.764 INFO:teuthology.orchestra.run.smithi055.stdout:====================================================================================================================== 2023-06-23T00:33:17.764 INFO:teuthology.orchestra.run.smithi055.stdout:Install 118 Packages 2023-06-23T00:33:17.764 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrade 2 Packages 2023-06-23T00:33:17.764 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:33:17.765 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 201 M 2023-06-23T00:33:17.765 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-06-23T00:33:18.305 INFO:teuthology.orchestra.run.smithi055.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 41 kB/s | 6.4 kB 00:00 2023-06-23T00:33:18.342 INFO:teuthology.orchestra.run.smithi141.stdout:(8/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86_ 10 MB/s | 17 MB 00:01 2023-06-23T00:33:18.453 INFO:teuthology.orchestra.run.smithi141.stdout:(9/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8. 11 MB/s | 17 MB 00:01 2023-06-23T00:33:18.454 INFO:teuthology.orchestra.run.smithi141.stdout:(10/120): ceph-selinux-17.2.6-719.g42ddc46e.el8 216 kB/s | 24 kB 00:00 2023-06-23T00:33:18.521 INFO:teuthology.orchestra.run.smithi141.stdout:(11/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 3.8 MB/s | 255 kB 00:00 2023-06-23T00:33:18.534 INFO:teuthology.orchestra.run.smithi055.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 3.7 MB/s | 873 kB 00:00 2023-06-23T00:33:18.574 INFO:teuthology.orchestra.run.smithi141.stdout:(12/120): libcephfs-devel-17.2.6-719.g42ddc46e. 467 kB/s | 25 kB 00:00 2023-06-23T00:33:18.582 INFO:teuthology.orchestra.run.smithi055.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 3.3 MB/s | 162 kB 00:00 2023-06-23T00:33:18.763 INFO:teuthology.orchestra.run.smithi141.stdout:(13/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 4.1 MB/s | 794 kB 00:00 2023-06-23T00:33:18.858 INFO:teuthology.orchestra.run.smithi141.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 1.8 MB/s | 174 kB 00:00 2023-06-23T00:33:18.955 INFO:teuthology.orchestra.run.smithi141.stdout:(15/120): librados-devel-17.2.6-719.g42ddc46e.e 1.3 MB/s | 130 kB 00:00 2023-06-23T00:33:19.218 INFO:teuthology.orchestra.run.smithi141.stdout:(16/120): libradosstriper1-17.2.6-719.g42ddc46e 1.9 MB/s | 506 kB 00:00 2023-06-23T00:33:19.467 INFO:teuthology.orchestra.run.smithi055.stdout:(4/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 2.6 MB/s | 2.3 MB 00:00 2023-06-23T00:33:19.920 INFO:teuthology.orchestra.run.smithi055.stdout:(5/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 2.9 MB/s | 5.2 MB 00:01 2023-06-23T00:33:20.138 INFO:teuthology.orchestra.run.smithi055.stdout:(6/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 2.3 MB/s | 1.6 MB 00:00 2023-06-23T00:33:20.569 INFO:teuthology.orchestra.run.smithi141.stdout:(17/120): ceph-common-17.2.6-719.g42ddc46e.el8. 5.2 MB/s | 23 MB 00:04 2023-06-23T00:33:20.612 INFO:teuthology.orchestra.run.smithi055.stdout:(7/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 5.7 MB/s | 3.9 MB 00:00 2023-06-23T00:33:20.627 INFO:teuthology.orchestra.run.smithi141.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 772 kB/s | 45 kB 00:00 2023-06-23T00:33:20.788 INFO:teuthology.orchestra.run.smithi141.stdout:(19/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 3.5 MB/s | 5.5 MB 00:01 2023-06-23T00:33:20.789 INFO:teuthology.orchestra.run.smithi141.stdout:(20/120): python3-ceph-common-17.2.6-719.g42ddc 713 kB/s | 115 kB 00:00 2023-06-23T00:33:20.842 INFO:teuthology.orchestra.run.smithi141.stdout:(21/120): python3-cephfs-17.2.6-719.g42ddc46e.e 3.4 MB/s | 190 kB 00:00 2023-06-23T00:33:20.966 INFO:teuthology.orchestra.run.smithi141.stdout:(22/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 2.8 MB/s | 357 kB 00:00 2023-06-23T00:33:21.023 INFO:teuthology.orchestra.run.smithi141.stdout:(23/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 1.8 MB/s | 107 kB 00:00 2023-06-23T00:33:21.075 INFO:teuthology.orchestra.run.smithi141.stdout:(24/120): python3-rados-17.2.6-719.g42ddc46e.el 1.3 MB/s | 371 kB 00:00 2023-06-23T00:33:21.079 INFO:teuthology.orchestra.run.smithi141.stdout:(25/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 1.6 MB/s | 91 kB 00:00 2023-06-23T00:33:21.142 INFO:teuthology.orchestra.run.smithi141.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 2.7 MB/s | 173 kB 00:00 2023-06-23T00:33:21.202 INFO:teuthology.orchestra.run.smithi141.stdout:(27/120): ceph-grafana-dashboards-17.2.6-719.g4 386 kB/s | 23 kB 00:00 2023-06-23T00:33:21.264 INFO:teuthology.orchestra.run.smithi141.stdout:(28/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 1.9 MB/s | 121 kB 00:00 2023-06-23T00:33:21.640 INFO:teuthology.orchestra.run.smithi141.stdout:(29/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 4.5 MB/s | 1.7 MB 00:00 2023-06-23T00:33:22.685 INFO:teuthology.orchestra.run.smithi141.stdout:(30/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 2.2 MB/s | 3.5 MB 00:01 2023-06-23T00:33:22.693 INFO:teuthology.orchestra.run.smithi055.stdout:(8/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8. 8.0 MB/s | 17 MB 00:02 2023-06-23T00:33:22.748 INFO:teuthology.orchestra.run.smithi141.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 3.7 MB/s | 238 kB 00:00 2023-06-23T00:33:22.767 INFO:teuthology.orchestra.run.smithi055.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 333 kB/s | 24 kB 00:00 2023-06-23T00:33:22.810 INFO:teuthology.orchestra.run.smithi141.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 776 kB/s | 47 kB 00:00 2023-06-23T00:33:22.895 INFO:teuthology.orchestra.run.smithi141.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 169 kB/s | 14 kB 00:00 2023-06-23T00:33:22.950 INFO:teuthology.orchestra.run.smithi141.stdout:(34/120): ceph-mgr-diskprediction-local-17.2.6- 5.6 MB/s | 7.4 MB 00:01 2023-06-23T00:33:23.019 INFO:teuthology.orchestra.run.smithi141.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 1.1 MB/s | 80 kB 00:00 2023-06-23T00:33:23.042 INFO:teuthology.orchestra.run.smithi141.stdout:(36/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 1.8 MB/s | 264 kB 00:00 2023-06-23T00:33:23.058 INFO:teuthology.orchestra.run.smithi141.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:23.061 INFO:teuthology.orchestra.run.smithi141.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:23.076 INFO:teuthology.orchestra.run.smithi141.stdout:(37/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 312 kB/s | 18 kB 00:00 2023-06-23T00:33:23.077 INFO:teuthology.orchestra.run.smithi141.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:23.078 INFO:teuthology.orchestra.run.smithi141.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:23.211 INFO:teuthology.orchestra.run.smithi141.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 438 kB/s | 59 kB 00:00 2023-06-23T00:33:23.212 INFO:teuthology.orchestra.run.smithi141.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:23.213 INFO:teuthology.orchestra.run.smithi141.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:23.268 INFO:teuthology.orchestra.run.smithi141.stdout:(39/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-06-23T00:33:23.269 INFO:teuthology.orchestra.run.smithi141.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm (IP: 8.43.85.72) 2023-06-23T00:33:23.277 INFO:teuthology.orchestra.run.smithi141.stdout:(40/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-06-23T00:33:23.314 INFO:teuthology.orchestra.run.smithi141.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.1 MB/s | 44 kB 00:00 2023-06-23T00:33:23.342 INFO:teuthology.orchestra.run.smithi141.stdout:(42/120): python3-asyncssh-2.7.0-2.el8.noarch.r 6.3 MB/s | 476 kB 00:00 2023-06-23T00:33:23.374 INFO:teuthology.orchestra.run.smithi141.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.0 MB/s | 33 kB 00:00 2023-06-23T00:33:23.378 INFO:teuthology.orchestra.run.smithi141.stdout:(44/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.9 MB/s | 185 kB 00:00 2023-06-23T00:33:23.405 INFO:teuthology.orchestra.run.smithi141.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 529 kB/s | 16 kB 00:00 2023-06-23T00:33:23.419 INFO:teuthology.orchestra.run.smithi141.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.1 MB/s | 173 kB 00:00 2023-06-23T00:33:23.448 INFO:teuthology.orchestra.run.smithi141.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 8.7 MB/s | 384 kB 00:00 2023-06-23T00:33:23.454 INFO:teuthology.orchestra.run.smithi141.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 2.3 MB/s | 81 kB 00:00 2023-06-23T00:33:23.479 INFO:teuthology.orchestra.run.smithi141.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 364 kB/s | 11 kB 00:00 2023-06-23T00:33:23.486 INFO:teuthology.orchestra.run.smithi141.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 568 kB/s | 18 kB 00:00 2023-06-23T00:33:23.519 INFO:teuthology.orchestra.run.smithi141.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-06-23T00:33:23.551 INFO:teuthology.orchestra.run.smithi141.stdout:(52/120): python3-kubernetes-11.0.0-6.el8.noarc 16 MB/s | 1.2 MB 00:00 2023-06-23T00:33:23.552 INFO:teuthology.orchestra.run.smithi141.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 1.7 MB/s | 59 kB 00:00 2023-06-23T00:33:23.582 INFO:teuthology.orchestra.run.smithi141.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.9 MB/s | 60 kB 00:00 2023-06-23T00:33:23.601 INFO:teuthology.orchestra.run.smithi141.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 5.7 MB/s | 283 kB 00:00 2023-06-23T00:33:23.613 INFO:teuthology.orchestra.run.smithi141.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 508 kB/s | 16 kB 00:00 2023-06-23T00:33:23.633 INFO:teuthology.orchestra.run.smithi141.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-06-23T00:33:23.647 INFO:teuthology.orchestra.run.smithi141.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 5.8 MB/s | 196 kB 00:00 2023-06-23T00:33:23.661 INFO:teuthology.orchestra.run.smithi055.stdout:(10/120): ceph-common-17.2.6-719.g42ddc46e.el8. 4.2 MB/s | 23 MB 00:05 2023-06-23T00:33:23.666 INFO:teuthology.orchestra.run.smithi141.stdout:(59/120): python3-rsa-4.8-1.el8.noarch.rpm 1.8 MB/s | 61 kB 00:00 2023-06-23T00:33:23.678 INFO:teuthology.orchestra.run.smithi141.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 603 kB/s | 19 kB 00:00 2023-06-23T00:33:23.697 INFO:teuthology.orchestra.run.smithi141.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 774 kB/s | 24 kB 00:00 2023-06-23T00:33:23.709 INFO:teuthology.orchestra.run.smithi141.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 957 kB/s | 29 kB 00:00 2023-06-23T00:33:23.729 INFO:teuthology.orchestra.run.smithi141.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 889 kB/s | 27 kB 00:00 2023-06-23T00:33:23.731 INFO:teuthology.orchestra.run.smithi055.stdout:(11/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 3.6 MB/s | 255 kB 00:00 2023-06-23T00:33:23.744 INFO:teuthology.orchestra.run.smithi141.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 7.4 MB/s | 255 kB 00:00 2023-06-23T00:33:23.773 INFO:teuthology.orchestra.run.smithi141.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 5.5 MB/s | 251 kB 00:00 2023-06-23T00:33:23.774 INFO:teuthology.orchestra.run.smithi141.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.9 MB/s | 61 kB 00:00 2023-06-23T00:33:23.791 INFO:teuthology.orchestra.run.smithi055.stdout:(12/120): libcephfs-devel-17.2.6-719.g42ddc46e. 416 kB/s | 25 kB 00:00 2023-06-23T00:33:23.808 INFO:teuthology.orchestra.run.smithi141.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 754 kB/s | 23 kB 00:00 2023-06-23T00:33:23.808 INFO:teuthology.orchestra.run.smithi141.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 2.4 MB/s | 85 kB 00:00 2023-06-23T00:33:23.812 INFO:teuthology.orchestra.run.smithi141.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 19 MB/s | 70 kB 00:00 2023-06-23T00:33:23.851 INFO:teuthology.orchestra.run.smithi141.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 483 kB/s | 19 kB 00:00 2023-06-23T00:33:23.867 INFO:teuthology.orchestra.run.smithi141.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.5 MB/s | 43 kB 00:00 2023-06-23T00:33:23.877 INFO:teuthology.orchestra.run.smithi141.stdout:(72/120): thrift-0.13.0-2.el8.x86_64.rpm 24 MB/s | 1.7 MB 00:00 2023-06-23T00:33:23.889 INFO:teuthology.orchestra.run.smithi141.stdout:(73/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.4 MB/s | 155 kB 00:00 2023-06-23T00:33:23.903 INFO:teuthology.orchestra.run.smithi141.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 2.7 MB/s | 48 kB 00:00 2023-06-23T00:33:23.917 INFO:teuthology.orchestra.run.smithi141.stdout:(75/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.4 MB/s | 220 kB 00:00 2023-06-23T00:33:23.923 INFO:teuthology.orchestra.run.smithi141.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 11 MB/s | 229 kB 00:00 2023-06-23T00:33:23.935 INFO:teuthology.orchestra.run.smithi141.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 2.4 MB/s | 43 kB 00:00 2023-06-23T00:33:23.952 INFO:teuthology.orchestra.run.smithi141.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.0 MB/s | 18 kB 00:00 2023-06-23T00:33:23.966 INFO:teuthology.orchestra.run.smithi141.stdout:(79/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 55 MB/s | 2.3 MB 00:00 2023-06-23T00:33:23.975 INFO:teuthology.orchestra.run.smithi141.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 25 MB/s | 605 kB 00:00 2023-06-23T00:33:23.985 INFO:teuthology.orchestra.run.smithi141.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 4.4 MB/s | 83 kB 00:00 2023-06-23T00:33:23.993 INFO:teuthology.orchestra.run.smithi141.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 3.2 MB/s | 59 kB 00:00 2023-06-23T00:33:23.995 INFO:teuthology.orchestra.run.smithi055.stdout:(13/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 3.8 MB/s | 794 kB 00:00 2023-06-23T00:33:24.004 INFO:teuthology.orchestra.run.smithi141.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 8.5 MB/s | 169 kB 00:00 2023-06-23T00:33:24.011 INFO:teuthology.orchestra.run.smithi141.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.2 MB/s | 38 kB 00:00 2023-06-23T00:33:24.024 INFO:teuthology.orchestra.run.smithi141.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.3 MB/s | 86 kB 00:00 2023-06-23T00:33:24.029 INFO:teuthology.orchestra.run.smithi141.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.2 MB/s | 58 kB 00:00 2023-06-23T00:33:24.043 INFO:teuthology.orchestra.run.smithi141.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.5 MB/s | 47 kB 00:00 2023-06-23T00:33:24.058 INFO:teuthology.orchestra.run.smithi141.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 19 MB/s | 559 kB 00:00 2023-06-23T00:33:24.065 INFO:teuthology.orchestra.run.smithi055.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 2.4 MB/s | 174 kB 00:00 2023-06-23T00:33:24.071 INFO:teuthology.orchestra.run.smithi141.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 23 MB/s | 639 kB 00:00 2023-06-23T00:33:24.076 INFO:teuthology.orchestra.run.smithi141.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 3.8 MB/s | 70 kB 00:00 2023-06-23T00:33:24.093 INFO:teuthology.orchestra.run.smithi141.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 8.9 MB/s | 200 kB 00:00 2023-06-23T00:33:24.103 INFO:teuthology.orchestra.run.smithi141.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 3.6 MB/s | 97 kB 00:00 2023-06-23T00:33:24.124 INFO:teuthology.orchestra.run.smithi055.stdout:(15/120): librados-devel-17.2.6-719.g42ddc46e.e 2.2 MB/s | 130 kB 00:00 2023-06-23T00:33:24.218 INFO:teuthology.orchestra.run.smithi055.stdout:(16/120): libradosstriper1-17.2.6-719.g42ddc46e 5.3 MB/s | 506 kB 00:00 2023-06-23T00:33:24.495 INFO:teuthology.orchestra.run.smithi141.stdout:(93/120): ceph-test-17.2.6-719.g42ddc46e.el8.x8 8.5 MB/s | 52 MB 00:06 2023-06-23T00:33:24.497 INFO:teuthology.orchestra.run.smithi141.stdout:(94/120): python3-prettytable-0.7.2-14.el8.noar 111 kB/s | 44 kB 00:00 2023-06-23T00:33:24.518 INFO:teuthology.orchestra.run.smithi141.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 7.8 MB/s | 161 kB 00:00 2023-06-23T00:33:24.532 INFO:teuthology.orchestra.run.smithi141.stdout:(96/120): python3-jsonpatch-1.21-2.el8.noarch.r 714 kB/s | 27 kB 00:00 2023-06-23T00:33:24.540 INFO:teuthology.orchestra.run.smithi141.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 14 MB/s | 320 kB 00:00 2023-06-23T00:33:24.556 INFO:teuthology.orchestra.run.smithi141.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 19 MB/s | 457 kB 00:00 2023-06-23T00:33:24.575 INFO:teuthology.orchestra.run.smithi141.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.1 MB/s | 39 kB 00:00 2023-06-23T00:33:24.604 INFO:teuthology.orchestra.run.smithi141.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 23 MB/s | 688 kB 00:00 2023-06-23T00:33:24.632 INFO:teuthology.orchestra.run.smithi141.stdout:(101/120): python3-babel-2.5.1-5.el8.noarch.rpm 52 MB/s | 4.8 MB 00:00 2023-06-23T00:33:24.636 INFO:teuthology.orchestra.run.smithi141.stdout:(102/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 7.9 MB/s | 259 kB 00:00 2023-06-23T00:33:24.653 INFO:teuthology.orchestra.run.smithi141.stdout:(103/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 7.4 MB/s | 126 kB 00:00 2023-06-23T00:33:24.655 INFO:teuthology.orchestra.run.smithi141.stdout:(104/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 9.7 MB/s | 225 kB 00:00 2023-06-23T00:33:24.673 INFO:teuthology.orchestra.run.smithi141.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 10 MB/s | 187 kB 00:00 2023-06-23T00:33:24.782 INFO:teuthology.orchestra.run.smithi141.stdout:(106/120): python3-scipy-1.0.0-20.module+el8.1. 21 MB/s | 15 MB 00:00 2023-06-23T00:33:24.790 INFO:teuthology.orchestra.run.smithi141.stdout:(107/120): python3-jinja2-2.10.1-2.el8_0.noarch 4.5 MB/s | 538 kB 00:00 2023-06-23T00:33:24.802 INFO:teuthology.orchestra.run.smithi141.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 15 MB/s | 302 kB 00:00 2023-06-23T00:33:24.808 INFO:teuthology.orchestra.run.smithi141.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 8.8 MB/s | 157 kB 00:00 2023-06-23T00:33:24.823 INFO:teuthology.orchestra.run.smithi141.stdout:(110/120): python3-numpy-1.14.3-9.el8.x86_64.rp 22 MB/s | 3.7 MB 00:00 2023-06-23T00:33:24.824 INFO:teuthology.orchestra.run.smithi141.stdout:(111/120): python3-pyasn1-modules-0.3.7-6.el8.n 4.9 MB/s | 110 kB 00:00 2023-06-23T00:33:24.827 INFO:teuthology.orchestra.run.smithi141.stdout:(112/120): librdkafka-0.11.4-1.el8.x86_64.rpm 18 MB/s | 353 kB 00:00 2023-06-23T00:33:24.839 INFO:teuthology.orchestra.run.smithi141.stdout:(113/120): python3-jsonpointer-1.10-11.el8.noar 1.2 MB/s | 20 kB 00:00 2023-06-23T00:33:24.845 INFO:teuthology.orchestra.run.smithi141.stdout:(114/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.1 MB/s | 54 kB 00:00 2023-06-23T00:33:24.849 INFO:teuthology.orchestra.run.smithi141.stdout:(115/120): libtool-2.4.6-25.el8.x86_64.rpm 29 MB/s | 709 kB 00:00 2023-06-23T00:33:24.875 INFO:teuthology.orchestra.run.smithi141.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 26 MB/s | 713 kB 00:00 2023-06-23T00:33:24.918 INFO:teuthology.orchestra.run.smithi141.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 60 MB/s | 4.7 MB 00:00 2023-06-23T00:33:24.938 INFO:teuthology.orchestra.run.smithi141.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 49 MB/s | 4.6 MB 00:00 2023-06-23T00:33:25.042 INFO:teuthology.orchestra.run.smithi055.stdout:(17/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 6.7 MB/s | 5.5 MB 00:00 2023-06-23T00:33:25.106 INFO:teuthology.orchestra.run.smithi055.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 700 kB/s | 45 kB 00:00 2023-06-23T00:33:25.182 INFO:teuthology.orchestra.run.smithi055.stdout:(19/120): python3-ceph-common-17.2.6-719.g42ddc 1.5 MB/s | 115 kB 00:00 2023-06-23T00:33:25.268 INFO:teuthology.orchestra.run.smithi055.stdout:(20/120): python3-cephfs-17.2.6-719.g42ddc46e.e 2.2 MB/s | 190 kB 00:00 2023-06-23T00:33:25.362 INFO:teuthology.orchestra.run.smithi055.stdout:(21/120): python3-rados-17.2.6-719.g42ddc46e.el 3.8 MB/s | 371 kB 00:00 2023-06-23T00:33:25.457 INFO:teuthology.orchestra.run.smithi055.stdout:(22/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 3.7 MB/s | 357 kB 00:00 2023-06-23T00:33:25.507 INFO:teuthology.orchestra.run.smithi141.stdout:(119/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 6.1 MB/s | 3.6 MB 00:00 2023-06-23T00:33:25.529 INFO:teuthology.orchestra.run.smithi055.stdout:(23/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 1.5 MB/s | 107 kB 00:00 2023-06-23T00:33:25.537 INFO:teuthology.orchestra.run.smithi141.stdout:(120/120): librados2-17.2.6-719.g42ddc46e.el8.x 5.2 MB/s | 3.5 MB 00:00 2023-06-23T00:33:25.541 INFO:teuthology.orchestra.run.smithi141.stdout:-------------------------------------------------------------------------------- 2023-06-23T00:33:25.541 INFO:teuthology.orchestra.run.smithi141.stdout:Total 20 MB/s | 201 MB 00:09 2023-06-23T00:33:25.571 INFO:teuthology.orchestra.run.smithi055.stdout:(24/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 2.1 MB/s | 91 kB 00:00 2023-06-23T00:33:25.588 INFO:teuthology.orchestra.run.smithi141.stdout:Running transaction check 2023-06-23T00:33:25.699 INFO:teuthology.orchestra.run.smithi141.stdout:Transaction check succeeded. 2023-06-23T00:33:25.700 INFO:teuthology.orchestra.run.smithi141.stdout:Running transaction test 2023-06-23T00:33:25.836 INFO:teuthology.orchestra.run.smithi055.stdout:(25/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86 2.9 MB/s | 17 MB 00:05 2023-06-23T00:33:25.867 INFO:teuthology.orchestra.run.smithi055.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 5.5 MB/s | 173 kB 00:00 2023-06-23T00:33:25.893 INFO:teuthology.orchestra.run.smithi055.stdout:(27/120): ceph-grafana-dashboards-17.2.6-719.g4 881 kB/s | 23 kB 00:00 2023-06-23T00:33:25.917 INFO:teuthology.orchestra.run.smithi055.stdout:(28/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 5.1 MB/s | 121 kB 00:00 2023-06-23T00:33:26.033 INFO:teuthology.orchestra.run.smithi055.stdout:(29/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 7.5 MB/s | 3.5 MB 00:00 2023-06-23T00:33:26.176 INFO:teuthology.orchestra.run.smithi055.stdout:(30/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 6.6 MB/s | 1.7 MB 00:00 2023-06-23T00:33:26.226 INFO:teuthology.orchestra.run.smithi055.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 4.6 MB/s | 238 kB 00:00 2023-06-23T00:33:26.247 INFO:teuthology.orchestra.run.smithi055.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 2.2 MB/s | 47 kB 00:00 2023-06-23T00:33:26.270 INFO:teuthology.orchestra.run.smithi055.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 628 kB/s | 14 kB 00:00 2023-06-23T00:33:26.323 INFO:teuthology.orchestra.run.smithi055.stdout:(34/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 5.0 MB/s | 264 kB 00:00 2023-06-23T00:33:26.349 INFO:teuthology.orchestra.run.smithi055.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 3.0 MB/s | 80 kB 00:00 2023-06-23T00:33:26.374 INFO:teuthology.orchestra.run.smithi055.stdout:(36/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 724 kB/s | 18 kB 00:00 2023-06-23T00:33:26.391 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:26.393 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:26.957 INFO:teuthology.orchestra.run.smithi055.stdout:(37/120): ceph-test-17.2.6-719.g42ddc46e.el8.x8 12 MB/s | 52 MB 00:04 2023-06-23T00:33:26.958 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:26.995 INFO:teuthology.orchestra.run.smithi055.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 493 kB/s | 306 kB 00:00 2023-06-23T00:33:26.995 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:26.996 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:26.997 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:27.050 INFO:teuthology.orchestra.run.smithi055.stdout:(39/120): ceph-mgr-diskprediction-local-17.2.6- 7.3 MB/s | 7.4 MB 00:01 2023-06-23T00:33:27.051 INFO:teuthology.orchestra.run.smithi055.stdout:(40/120): liboath-2.6.2-3.el8.x86_64.rpm 632 kB/s | 59 kB 00:00 2023-06-23T00:33:27.051 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm (IP: 8.43.85.72) 2023-06-23T00:33:27.069 INFO:teuthology.orchestra.run.smithi055.stdout:[MIRROR] python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T00:33:27.135 INFO:teuthology.orchestra.run.smithi055.stdout:(41/120): libunwind-1.3.1-3.el8.x86_64.rpm 537 kB/s | 75 kB 00:00 2023-06-23T00:33:27.146 INFO:teuthology.orchestra.run.smithi055.stdout:(42/120): python3-asyncssh-2.7.0-2.el8.noarch.r 4.8 MB/s | 476 kB 00:00 2023-06-23T00:33:27.177 INFO:teuthology.orchestra.run.smithi055.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.1 MB/s | 33 kB 00:00 2023-06-23T00:33:27.186 INFO:teuthology.orchestra.run.smithi055.stdout:(44/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 321 kB/s | 44 kB 00:00 2023-06-23T00:33:27.206 INFO:teuthology.orchestra.run.smithi055.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 557 kB/s | 16 kB 00:00 2023-06-23T00:33:27.221 INFO:teuthology.orchestra.run.smithi055.stdout:(46/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.1 MB/s | 185 kB 00:00 2023-06-23T00:33:27.249 INFO:teuthology.orchestra.run.smithi055.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 8.8 MB/s | 384 kB 00:00 2023-06-23T00:33:27.259 INFO:teuthology.orchestra.run.smithi055.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 2.1 MB/s | 81 kB 00:00 2023-06-23T00:33:27.280 INFO:teuthology.orchestra.run.smithi055.stdout:(49/120): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-06-23T00:33:27.281 INFO:teuthology.orchestra.run.smithi055.stdout:(50/120): python3-jaraco-6.2-6.el8.noarch.rpm 355 kB/s | 11 kB 00:00 2023-06-23T00:33:27.290 INFO:teuthology.orchestra.run.smithi055.stdout:(51/120): python3-jaraco-functools-2.0-4.el8.no 571 kB/s | 18 kB 00:00 2023-06-23T00:33:27.315 INFO:teuthology.orchestra.run.smithi055.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-06-23T00:33:27.325 INFO:teuthology.orchestra.run.smithi055.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 1.7 MB/s | 59 kB 00:00 2023-06-23T00:33:27.350 INFO:teuthology.orchestra.run.smithi055.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.7 MB/s | 60 kB 00:00 2023-06-23T00:33:27.354 INFO:teuthology.orchestra.run.smithi055.stdout:(55/120): python3-kubernetes-11.0.0-6.el8.noarc 16 MB/s | 1.2 MB 00:00 2023-06-23T00:33:27.380 INFO:teuthology.orchestra.run.smithi055.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 530 kB/s | 16 kB 00:00 2023-06-23T00:33:27.384 INFO:teuthology.orchestra.run.smithi055.stdout:(57/120): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-06-23T00:33:27.384 INFO:teuthology.orchestra.run.smithi055.stdout:(58/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.1 MB/s | 34 kB 00:00 2023-06-23T00:33:27.396 INFO:teuthology.orchestra.run.smithi141.stdout:Transaction test succeeded. 2023-06-23T00:33:27.401 INFO:teuthology.orchestra.run.smithi141.stdout:Running transaction 2023-06-23T00:33:27.414 INFO:teuthology.orchestra.run.smithi055.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 637 kB/s | 19 kB 00:00 2023-06-23T00:33:27.415 INFO:teuthology.orchestra.run.smithi055.stdout:(60/120): python3-rsa-4.8-1.el8.noarch.rpm 2.0 MB/s | 61 kB 00:00 2023-06-23T00:33:27.430 INFO:teuthology.orchestra.run.smithi055.stdout:(61/120): python3-routes-2.4.1-12.el8.noarch.rp 3.9 MB/s | 196 kB 00:00 2023-06-23T00:33:27.444 INFO:teuthology.orchestra.run.smithi055.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 814 kB/s | 24 kB 00:00 2023-06-23T00:33:27.444 INFO:teuthology.orchestra.run.smithi055.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 995 kB/s | 29 kB 00:00 2023-06-23T00:33:27.461 INFO:teuthology.orchestra.run.smithi055.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 900 kB/s | 27 kB 00:00 2023-06-23T00:33:27.477 INFO:teuthology.orchestra.run.smithi055.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 7.5 MB/s | 255 kB 00:00 2023-06-23T00:33:27.483 INFO:teuthology.orchestra.run.smithi055.stdout:(66/120): python3-webob-1.8.5-1.el8.1.noarch.rp 6.4 MB/s | 251 kB 00:00 2023-06-23T00:33:27.495 INFO:teuthology.orchestra.run.smithi055.stdout:(67/120): python3-websocket-client-0.56.0-5.el8 1.7 MB/s | 61 kB 00:00 2023-06-23T00:33:27.507 INFO:teuthology.orchestra.run.smithi055.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 2.8 MB/s | 85 kB 00:00 2023-06-23T00:33:27.512 INFO:teuthology.orchestra.run.smithi055.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 14 MB/s | 70 kB 00:00 2023-06-23T00:33:27.513 INFO:teuthology.orchestra.run.smithi055.stdout:(70/120): python3-zc-lockfile-2.0-2.el8.noarch. 767 kB/s | 23 kB 00:00 2023-06-23T00:33:27.555 INFO:teuthology.orchestra.run.smithi055.stdout:(71/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 440 kB/s | 19 kB 00:00 2023-06-23T00:33:27.565 INFO:teuthology.orchestra.run.smithi055.stdout:(72/120): python3-jwt-1.6.1-2.el8.noarch.rpm 825 kB/s | 43 kB 00:00 2023-06-23T00:33:27.577 INFO:teuthology.orchestra.run.smithi055.stdout:(73/120): python3-oauthlib-2.1.0-1.el8.noarch.r 6.9 MB/s | 155 kB 00:00 2023-06-23T00:33:27.587 INFO:teuthology.orchestra.run.smithi055.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 9.7 MB/s | 220 kB 00:00 2023-06-23T00:33:27.598 INFO:teuthology.orchestra.run.smithi055.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 2.2 MB/s | 48 kB 00:00 2023-06-23T00:33:27.609 INFO:teuthology.orchestra.run.smithi055.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 11 MB/s | 229 kB 00:00 2023-06-23T00:33:27.627 INFO:teuthology.orchestra.run.smithi055.stdout:(77/120): thrift-0.13.0-2.el8.x86_64.rpm 13 MB/s | 1.7 MB 00:00 2023-06-23T00:33:27.629 INFO:teuthology.orchestra.run.smithi055.stdout:(78/120): python3-requests-oauthlib-1.0.0-1.el8 1.4 MB/s | 43 kB 00:00 2023-06-23T00:33:27.659 INFO:teuthology.orchestra.run.smithi055.stdout:(79/120): pcre2-devel-10.32-2.el8.x86_64.rpm 20 MB/s | 605 kB 00:00 2023-06-23T00:33:27.666 INFO:teuthology.orchestra.run.smithi055.stdout:(80/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 454 kB/s | 18 kB 00:00 2023-06-23T00:33:27.674 INFO:teuthology.orchestra.run.smithi055.stdout:(81/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 36 MB/s | 2.3 MB 00:00 2023-06-23T00:33:27.676 INFO:teuthology.orchestra.run.smithi055.stdout:(82/120): fuse-2.9.7-12.el8.x86_64.rpm 4.8 MB/s | 83 kB 00:00 2023-06-23T00:33:27.693 INFO:teuthology.orchestra.run.smithi055.stdout:(83/120): elfutils-libelf-devel-0.182-3.el8.x86 2.2 MB/s | 59 kB 00:00 2023-06-23T00:33:27.698 INFO:teuthology.orchestra.run.smithi055.stdout:(84/120): libquadmath-8.4.1-1.el8.x86_64.rpm 7.1 MB/s | 169 kB 00:00 2023-06-23T00:33:27.713 INFO:teuthology.orchestra.run.smithi055.stdout:(85/120): libcom_err-devel-1.45.6-1.el8.x86_64. 1.0 MB/s | 38 kB 00:00 2023-06-23T00:33:27.723 INFO:teuthology.orchestra.run.smithi055.stdout:(86/120): libsepol-devel-2.9-2.el8.x86_64.rpm 2.8 MB/s | 86 kB 00:00 2023-06-23T00:33:27.730 INFO:teuthology.orchestra.run.smithi055.stdout:(87/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2023-06-23T00:33:27.742 INFO:teuthology.orchestra.run.smithi055.stdout:(88/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.6 MB/s | 47 kB 00:00 2023-06-23T00:33:27.751 INFO:teuthology.orchestra.run.smithi055.stdout:(89/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 20 MB/s | 559 kB 00:00 2023-06-23T00:33:27.778 INFO:teuthology.orchestra.run.smithi055.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 1.9 MB/s | 70 kB 00:00 2023-06-23T00:33:27.782 INFO:teuthology.orchestra.run.smithi055.stdout:(91/120): libgfortran-8.4.1-1.el8.x86_64.rpm 12 MB/s | 639 kB 00:00 2023-06-23T00:33:27.785 INFO:teuthology.orchestra.run.smithi055.stdout:(92/120): libselinux-devel-2.9-5.el8.x86_64.rpm 5.8 MB/s | 200 kB 00:00 2023-06-23T00:33:27.799 INFO:teuthology.orchestra.run.smithi055.stdout:(93/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 4.5 MB/s | 97 kB 00:00 2023-06-23T00:33:27.809 INFO:teuthology.orchestra.run.smithi055.stdout:(94/120): python3-prettytable-0.7.2-14.el8.noar 1.9 MB/s | 44 kB 00:00 2023-06-23T00:33:27.819 INFO:teuthology.orchestra.run.smithi055.stdout:(95/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.3 MB/s | 27 kB 00:00 2023-06-23T00:33:27.832 INFO:teuthology.orchestra.run.smithi055.stdout:(96/120): jq-1.5-12.el8.x86_64.rpm 6.9 MB/s | 161 kB 00:00 2023-06-23T00:33:27.847 INFO:teuthology.orchestra.run.smithi055.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 11 MB/s | 320 kB 00:00 2023-06-23T00:33:27.857 INFO:teuthology.orchestra.run.smithi055.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 18 MB/s | 457 kB 00:00 2023-06-23T00:33:27.875 INFO:teuthology.orchestra.run.smithi055.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.1 MB/s | 39 kB 00:00 2023-06-23T00:33:27.907 INFO:teuthology.orchestra.run.smithi055.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 21 MB/s | 688 kB 00:00 2023-06-23T00:33:27.929 INFO:teuthology.orchestra.run.smithi055.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 12 MB/s | 259 kB 00:00 2023-06-23T00:33:27.950 INFO:teuthology.orchestra.run.smithi055.stdout:(102/120): python3-babel-2.5.1-5.el8.noarch.rpm 47 MB/s | 4.8 MB 00:00 2023-06-23T00:33:27.954 INFO:teuthology.orchestra.run.smithi055.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 9.1 MB/s | 225 kB 00:00 2023-06-23T00:33:27.970 INFO:teuthology.orchestra.run.smithi055.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 6.2 MB/s | 126 kB 00:00 2023-06-23T00:33:27.994 INFO:teuthology.orchestra.run.smithi055.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 7.8 MB/s | 187 kB 00:00 2023-06-23T00:33:28.124 INFO:teuthology.orchestra.run.smithi055.stdout:(106/120): python3-scipy-1.0.0-20.module+el8.1. 43 MB/s | 15 MB 00:00 2023-06-23T00:33:28.130 INFO:teuthology.orchestra.run.smithi055.stdout:(107/120): python3-jinja2-2.10.1-2.el8_0.noarch 3.9 MB/s | 538 kB 00:00 2023-06-23T00:33:28.143 INFO:teuthology.orchestra.run.smithi055.stdout:(108/120): python3-numpy-1.14.3-9.el8.x86_64.rp 20 MB/s | 3.7 MB 00:00 2023-06-23T00:33:28.146 INFO:teuthology.orchestra.run.smithi055.stdout:(109/120): socat-1.7.3.3-2.el8.x86_64.rpm 12 MB/s | 302 kB 00:00 2023-06-23T00:33:28.150 INFO:teuthology.orchestra.run.smithi055.stdout:(110/120): python3-mako-1.0.6-13.el8.noarch.rpm 7.7 MB/s | 157 kB 00:00 2023-06-23T00:33:28.161 INFO:teuthology.orchestra.run.smithi055.stdout:(111/120): python3-pyasn1-modules-0.3.7-6.el8.n 5.8 MB/s | 110 kB 00:00 2023-06-23T00:33:28.169 INFO:teuthology.orchestra.run.smithi055.stdout:(112/120): librdkafka-0.11.4-1.el8.x86_64.rpm 16 MB/s | 353 kB 00:00 2023-06-23T00:33:28.173 INFO:teuthology.orchestra.run.smithi055.stdout:(113/120): python3-jsonpointer-1.10-11.el8.noar 915 kB/s | 20 kB 00:00 2023-06-23T00:33:28.188 INFO:teuthology.orchestra.run.smithi055.stdout:(114/120): python3-pytz-2017.2-9.el8.noarch.rpm 2.8 MB/s | 54 kB 00:00 2023-06-23T00:33:28.195 INFO:teuthology.orchestra.run.smithi055.stdout:(115/120): libtool-2.4.6-25.el8.x86_64.rpm 21 MB/s | 709 kB 00:00 2023-06-23T00:33:28.225 INFO:teuthology.orchestra.run.smithi055.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 23 MB/s | 713 kB 00:00 2023-06-23T00:33:28.268 INFO:teuthology.orchestra.run.smithi055.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 50 MB/s | 4.7 MB 00:00 2023-06-23T00:33:28.292 INFO:teuthology.orchestra.run.smithi055.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 44 MB/s | 4.6 MB 00:00 2023-06-23T00:33:28.672 INFO:teuthology.orchestra.run.smithi055.stdout:(119/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 8.9 MB/s | 3.6 MB 00:00 2023-06-23T00:33:28.682 INFO:teuthology.orchestra.run.smithi055.stdout:(120/120): librados2-17.2.6-719.g42ddc46e.el8.x 7.6 MB/s | 3.5 MB 00:00 2023-06-23T00:33:28.686 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-06-23T00:33:28.686 INFO:teuthology.orchestra.run.smithi055.stdout:Total 18 MB/s | 201 MB 00:10 2023-06-23T00:33:28.739 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-06-23T00:33:28.864 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-06-23T00:33:28.865 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-06-23T00:33:30.584 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-06-23T00:33:30.588 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-06-23T00:33:32.065 INFO:teuthology.orchestra.run.smithi141.stdout: Preparing : 1/1 2023-06-23T00:33:32.532 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-23T00:33:32.758 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-23T00:33:32.882 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-23T00:33:33.100 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-23T00:33:33.466 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-23T00:33:33.674 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T00:33:34.115 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T00:33:34.300 INFO:teuthology.orchestra.run.smithi141.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T00:33:34.431 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T00:33:34.616 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T00:33:34.725 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T00:33:34.971 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T00:33:35.313 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-06-23T00:33:35.438 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-06-23T00:33:35.695 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-23T00:33:35.894 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-23T00:33:36.008 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-23T00:33:36.017 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-23T00:33:36.207 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T00:33:36.219 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-23T00:33:36.309 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T00:33:36.577 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-23T00:33:36.655 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-23T00:33:36.756 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-23T00:33:36.919 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T00:33:36.966 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T00:33:37.076 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T00:33:37.249 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T00:33:37.347 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T00:33:37.405 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T00:33:37.536 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T00:33:37.704 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T00:33:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T00:33:37.966 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-06-23T00:33:38.039 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T00:33:38.231 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-06-23T00:33:38.241 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T00:33:38.489 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-06-23T00:33:38.617 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-06-23T00:33:38.685 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-06-23T00:33:38.832 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-23T00:33:38.978 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-23T00:33:39.157 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-23T00:33:39.337 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-23T00:33:39.552 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T00:33:39.658 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T00:33:39.707 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-23T00:33:39.865 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-23T00:33:39.924 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-23T00:33:40.119 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-23T00:33:40.382 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-23T00:33:40.540 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-06-23T00:33:40.922 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T00:33:41.041 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T00:33:41.227 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T00:33:41.291 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-06-23T00:33:41.313 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T00:33:41.520 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-06-23T00:33:41.593 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-06-23T00:33:41.854 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-06-23T00:33:41.867 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-06-23T00:33:42.093 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-06-23T00:33:42.185 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-23T00:33:42.261 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-06-23T00:33:42.396 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-23T00:33:42.402 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-23T00:33:42.534 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-23T00:33:42.549 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-23T00:33:42.712 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-23T00:33:42.816 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T00:33:43.309 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-23T00:33:43.324 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-06-23T00:33:43.493 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-23T00:33:43.649 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-06-23T00:33:43.889 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-23T00:33:44.073 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-23T00:33:44.277 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-06-23T00:33:45.126 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-06-23T00:33:45.416 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-06-23T00:33:45.702 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-06-23T00:33:45.960 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-23T00:33:45.980 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-23T00:33:46.342 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-23T00:33:46.354 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-06-23T00:33:46.523 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-23T00:33:46.836 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T00:33:47.112 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-06-23T00:33:47.395 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-06-23T00:33:47.430 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-06-23T00:33:47.651 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-23T00:33:47.744 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-06-23T00:33:48.018 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-06-23T00:33:48.253 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-23T00:33:48.450 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-23T00:33:48.713 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-23T00:33:48.910 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-23T00:33:49.208 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-23T00:33:49.445 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-06-23T00:33:49.662 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-06-23T00:33:49.852 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-06-23T00:33:49.891 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-23T00:33:50.067 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-06-23T00:33:50.230 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-23T00:33:50.275 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-06-23T00:33:50.312 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-23T00:33:50.497 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-06-23T00:33:50.698 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-06-23T00:33:50.878 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-06-23T00:33:51.059 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-06-23T00:33:51.223 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-06-23T00:33:51.404 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-06-23T00:33:51.440 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-06-23T00:33:51.592 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-06-23T00:33:51.827 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-06-23T00:33:52.000 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-06-23T00:33:52.180 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-06-23T00:33:52.350 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-06-23T00:33:52.475 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-06-23T00:33:52.627 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-06-23T00:33:52.769 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-23T00:33:52.905 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-06-23T00:33:53.004 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-23T00:33:53.203 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-06-23T00:33:53.256 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-23T00:33:53.363 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-23T00:33:53.481 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-23T00:33:53.568 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-23T00:33:53.762 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-23T00:33:53.818 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-23T00:33:54.025 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-06-23T00:33:54.046 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-23T00:33:54.258 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-06-23T00:33:54.261 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-23T00:33:54.431 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-23T00:33:54.475 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-06-23T00:33:54.616 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-23T00:33:54.705 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-06-23T00:33:54.823 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-23T00:33:54.892 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-23T00:33:54.972 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-23T00:33:55.084 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-23T00:33:55.185 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-06-23T00:33:55.334 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-23T00:33:55.377 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-06-23T00:33:55.516 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-23T00:33:55.595 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-06-23T00:33:55.694 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-23T00:33:55.740 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-06-23T00:33:55.944 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-06-23T00:33:55.955 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-23T00:33:56.136 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-06-23T00:33:56.175 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-23T00:33:56.273 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-06-23T00:33:56.402 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-23T00:33:56.512 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-06-23T00:33:56.697 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-06-23T00:33:56.703 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-23T00:33:56.962 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-23T00:33:57.086 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-06-23T00:33:57.164 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-23T00:33:57.385 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-06-23T00:33:57.442 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-23T00:33:57.660 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-06-23T00:33:57.968 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-06-23T00:33:58.135 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-23T00:33:58.351 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-23T00:33:58.613 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-23T00:33:58.652 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-23T00:33:58.885 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-23T00:33:58.986 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-23T00:33:59.109 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-23T00:33:59.222 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-23T00:33:59.298 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-23T00:33:59.448 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-23T00:33:59.587 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-23T00:33:59.792 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-23T00:33:59.882 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-23T00:34:00.017 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-23T00:34:00.236 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-23T00:34:00.251 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-23T00:34:00.479 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-23T00:34:00.496 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-23T00:34:00.624 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-23T00:34:00.704 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-23T00:34:00.920 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-23T00:34:00.927 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-23T00:34:01.181 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-23T00:34:01.433 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-23T00:34:01.757 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-23T00:34:02.059 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-23T00:34:02.288 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-23T00:34:02.620 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-23T00:34:03.867 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-23T00:34:04.202 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-23T00:34:04.429 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-23T00:34:04.798 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-23T00:34:05.113 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-23T00:34:05.445 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-23T00:34:05.710 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-23T00:34:05.931 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T00:34:05.960 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-23T00:34:06.199 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-23T00:34:06.202 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T00:34:06.801 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T00:34:07.119 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T00:34:07.227 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T00:34:07.385 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T00:34:11.470 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T00:34:11.690 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T00:34:12.330 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T00:34:12.936 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T00:34:13.419 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T00:34:13.573 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T00:34:25.262 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T00:34:25.262 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /sys 2023-06-23T00:34:25.262 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /proc 2023-06-23T00:34:25.262 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /dev 2023-06-23T00:34:25.262 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /run 2023-06-23T00:34:25.262 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /mnt 2023-06-23T00:34:25.263 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /var/tmp 2023-06-23T00:34:25.263 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /home 2023-06-23T00:34:25.263 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /tmp 2023-06-23T00:34:25.263 INFO:teuthology.orchestra.run.smithi141.stdout:skipping the directory /dev 2023-06-23T00:34:25.263 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:34:25.632 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T00:34:27.119 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T00:34:27.290 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T00:34:27.512 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T00:34:27.674 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T00:34:28.064 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T00:34:28.232 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T00:34:28.248 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T00:34:31.031 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T00:34:31.242 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T00:34:31.379 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-23T00:34:31.586 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-23T00:34:31.747 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T00:34:31.964 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T00:34:31.984 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T00:34:31.984 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /sys 2023-06-23T00:34:31.985 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /proc 2023-06-23T00:34:31.985 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /dev 2023-06-23T00:34:31.985 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /run 2023-06-23T00:34:31.985 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /mnt 2023-06-23T00:34:31.985 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /var/tmp 2023-06-23T00:34:31.985 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /home 2023-06-23T00:34:31.985 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /tmp 2023-06-23T00:34:31.986 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /dev 2023-06-23T00:34:31.986 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:34:32.169 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T00:34:32.379 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T00:34:33.142 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T00:34:33.330 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T00:34:33.429 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T00:34:33.829 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-23T00:34:33.960 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T00:34:34.056 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T00:34:34.093 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T00:34:34.148 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T00:34:34.280 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T00:34:34.291 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T00:34:34.384 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T00:34:34.546 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T00:34:34.961 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T00:34:35.162 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T00:34:35.180 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T00:34:35.887 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-23T00:34:36.147 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T00:34:36.221 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T00:34:36.405 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T00:34:36.448 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T00:34:36.655 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T00:34:37.004 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T00:34:37.180 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T00:34:38.231 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T00:34:38.464 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T00:34:38.635 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-23T00:34:38.826 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-23T00:34:39.036 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T00:34:39.269 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T00:34:39.505 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T00:34:40.540 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T00:34:40.736 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T00:34:40.843 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T00:34:41.262 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-23T00:34:41.497 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T00:34:41.537 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T00:34:41.736 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T00:34:41.748 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T00:34:42.113 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T00:34:43.087 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-23T00:34:43.421 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-23T00:34:43.490 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-06-23T00:34:43.664 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T00:34:43.683 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 110/122 2023-06-23T00:34:43.789 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T00:34:43.914 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-06-23T00:34:43.936 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T00:34:43.981 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T00:34:44.112 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-06-23T00:34:44.123 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-06-23T00:34:44.161 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T00:34:44.275 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-23T00:34:44.442 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-23T00:34:44.532 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T00:34:44.595 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-23T00:34:44.694 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T00:34:44.820 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-23T00:34:45.093 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-23T00:34:45.320 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-06-23T00:34:45.382 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-06-23T00:34:45.612 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-06-23T00:34:45.621 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-06-23T00:34:45.772 INFO:teuthology.orchestra.run.smithi141.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-06-23T00:34:45.772 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T00:34:45.982 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T00:34:45.982 INFO:teuthology.orchestra.run.smithi141.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T00:34:46.512 INFO:teuthology.orchestra.run.smithi141.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T00:34:46.512 INFO:teuthology.orchestra.run.smithi141.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T00:34:46.512 INFO:teuthology.orchestra.run.smithi141.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T00:34:46.512 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:34:46.894 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-23T00:34:46.895 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-23T00:34:46.895 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-23T00:34:46.895 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T00:34:46.896 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-23T00:34:46.896 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T00:34:46.896 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T00:34:46.896 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-23T00:34:46.897 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-23T00:34:46.897 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T00:34:46.897 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-23T00:34:46.897 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-23T00:34:46.898 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T00:34:46.898 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-23T00:34:46.898 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-23T00:34:46.899 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-23T00:34:46.899 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-23T00:34:46.899 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-23T00:34:46.899 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-23T00:34:46.900 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-23T00:34:46.900 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-23T00:34:46.900 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-23T00:34:46.900 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-23T00:34:46.901 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-23T00:34:46.901 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-23T00:34:46.901 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-23T00:34:46.901 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-23T00:34:46.902 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T00:34:46.902 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-23T00:34:46.902 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-23T00:34:46.902 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-23T00:34:46.903 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-23T00:34:46.903 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-23T00:34:46.903 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-23T00:34:46.904 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-23T00:34:46.904 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-23T00:34:46.904 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-23T00:34:46.904 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-23T00:34:46.905 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-23T00:34:46.905 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-23T00:34:46.905 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-23T00:34:46.905 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-23T00:34:46.906 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-23T00:34:46.906 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-23T00:34:46.906 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-23T00:34:46.906 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-23T00:34:46.907 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-23T00:34:46.907 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-23T00:34:46.907 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-23T00:34:46.907 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-23T00:34:46.908 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-23T00:34:46.908 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-23T00:34:46.908 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-23T00:34:46.909 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-23T00:34:46.909 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-23T00:34:46.909 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-23T00:34:46.909 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-23T00:34:46.910 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-23T00:34:46.910 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-23T00:34:46.910 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-23T00:34:46.910 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-23T00:34:46.911 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-23T00:34:46.911 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-23T00:34:46.911 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-23T00:34:46.911 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-23T00:34:46.912 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-23T00:34:46.912 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-23T00:34:46.912 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-23T00:34:46.912 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-23T00:34:46.913 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-23T00:34:46.913 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-23T00:34:46.913 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-06-23T00:34:46.914 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-06-23T00:34:46.914 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-06-23T00:34:46.914 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-06-23T00:34:46.914 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-06-23T00:34:46.915 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-06-23T00:34:46.915 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-06-23T00:34:46.915 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-06-23T00:34:46.915 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-06-23T00:34:46.916 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-06-23T00:34:46.916 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-06-23T00:34:46.917 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-06-23T00:34:46.917 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-06-23T00:34:46.918 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-06-23T00:34:46.918 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-06-23T00:34:46.918 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-06-23T00:34:46.919 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-06-23T00:34:46.919 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-06-23T00:34:46.920 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-06-23T00:34:46.920 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-06-23T00:34:46.920 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-06-23T00:34:46.921 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-06-23T00:34:46.921 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-06-23T00:34:46.921 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-06-23T00:34:46.921 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-06-23T00:34:46.921 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-06-23T00:34:46.921 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-06-23T00:34:46.921 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-06-23T00:34:46.922 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-06-23T00:34:46.922 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-06-23T00:34:46.922 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-06-23T00:34:46.922 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-06-23T00:34:46.922 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-06-23T00:34:46.922 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-06-23T00:34:46.923 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-06-23T00:34:46.923 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-06-23T00:34:46.923 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-06-23T00:34:46.923 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-06-23T00:34:46.923 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-06-23T00:34:46.923 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-06-23T00:34:46.923 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-06-23T00:34:46.923 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-06-23T00:34:46.924 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-06-23T00:34:46.924 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-06-23T00:34:46.924 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-06-23T00:34:46.924 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-06-23T00:34:46.924 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-06-23T00:34:46.924 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-23T00:34:46.924 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-23T00:34:46.925 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-23T00:34:49.097 INFO:teuthology.orchestra.run.smithi141.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T00:34:49.097 INFO:teuthology.orchestra.run.smithi141.stdout:Last metadata expiration check: 0:01:42 ago on Fri 23 Jun 2023 12:33:07 AM UTC. 2023-06-23T00:34:49.972 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T00:34:51.005 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-23T00:34:51.274 INFO:teuthology.orchestra.run.smithi141.stdout:Installed products updated. 2023-06-23T00:34:51.378 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-06-23T00:34:51.418 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:34:51.419 INFO:teuthology.orchestra.run.smithi141.stdout:Upgraded: 2023-06-23T00:34:51.419 INFO:teuthology.orchestra.run.smithi141.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.419 INFO:teuthology.orchestra.run.smithi141.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.419 INFO:teuthology.orchestra.run.smithi141.stdout:Installed: 2023-06-23T00:34:51.420 INFO:teuthology.orchestra.run.smithi141.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T00:34:51.420 INFO:teuthology.orchestra.run.smithi141.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T00:34:51.420 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.420 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.420 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.421 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.421 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:51.421 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.421 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.422 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.422 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:51.422 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:51.422 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:51.422 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:51.423 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:51.423 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.423 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.423 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:51.423 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.423 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.423 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.424 INFO:teuthology.orchestra.run.smithi141.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:51.424 INFO:teuthology.orchestra.run.smithi141.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:51.424 INFO:teuthology.orchestra.run.smithi141.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.424 INFO:teuthology.orchestra.run.smithi141.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:51.424 INFO:teuthology.orchestra.run.smithi141.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-23T00:34:51.424 INFO:teuthology.orchestra.run.smithi141.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T00:34:51.424 INFO:teuthology.orchestra.run.smithi141.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-23T00:34:51.425 INFO:teuthology.orchestra.run.smithi141.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T00:34:51.425 INFO:teuthology.orchestra.run.smithi141.stdout: jq-1.5-12.el8.x86_64 2023-06-23T00:34:51.425 INFO:teuthology.orchestra.run.smithi141.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-23T00:34:51.425 INFO:teuthology.orchestra.run.smithi141.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-23T00:34:51.425 INFO:teuthology.orchestra.run.smithi141.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T00:34:51.425 INFO:teuthology.orchestra.run.smithi141.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.425 INFO:teuthology.orchestra.run.smithi141.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.425 INFO:teuthology.orchestra.run.smithi141.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.426 INFO:teuthology.orchestra.run.smithi141.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-23T00:34:51.426 INFO:teuthology.orchestra.run.smithi141.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-23T00:34:51.426 INFO:teuthology.orchestra.run.smithi141.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T00:34:51.426 INFO:teuthology.orchestra.run.smithi141.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-23T00:34:51.426 INFO:teuthology.orchestra.run.smithi141.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T00:34:51.426 INFO:teuthology.orchestra.run.smithi141.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.426 INFO:teuthology.orchestra.run.smithi141.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.427 INFO:teuthology.orchestra.run.smithi141.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-23T00:34:51.427 INFO:teuthology.orchestra.run.smithi141.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.427 INFO:teuthology.orchestra.run.smithi141.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T00:34:51.427 INFO:teuthology.orchestra.run.smithi141.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-23T00:34:51.427 INFO:teuthology.orchestra.run.smithi141.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T00:34:51.427 INFO:teuthology.orchestra.run.smithi141.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T00:34:51.427 INFO:teuthology.orchestra.run.smithi141.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-23T00:34:51.427 INFO:teuthology.orchestra.run.smithi141.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T00:34:51.428 INFO:teuthology.orchestra.run.smithi141.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T00:34:51.428 INFO:teuthology.orchestra.run.smithi141.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T00:34:51.428 INFO:teuthology.orchestra.run.smithi141.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-23T00:34:51.428 INFO:teuthology.orchestra.run.smithi141.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-23T00:34:51.428 INFO:teuthology.orchestra.run.smithi141.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-23T00:34:51.428 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T00:34:51.428 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T00:34:51.429 INFO:teuthology.orchestra.run.smithi141.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T00:34:51.429 INFO:teuthology.orchestra.run.smithi141.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T00:34:51.429 INFO:teuthology.orchestra.run.smithi141.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-23T00:34:51.429 INFO:teuthology.orchestra.run.smithi141.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T00:34:51.429 INFO:teuthology.orchestra.run.smithi141.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T00:34:51.429 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T00:34:51.429 INFO:teuthology.orchestra.run.smithi141.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.429 INFO:teuthology.orchestra.run.smithi141.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.430 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.430 INFO:teuthology.orchestra.run.smithi141.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T00:34:51.430 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T00:34:51.430 INFO:teuthology.orchestra.run.smithi141.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T00:34:51.430 INFO:teuthology.orchestra.run.smithi141.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T00:34:51.430 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T00:34:51.431 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T00:34:51.431 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-23T00:34:51.431 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T00:34:51.431 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T00:34:51.431 INFO:teuthology.orchestra.run.smithi141.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T00:34:51.431 INFO:teuthology.orchestra.run.smithi141.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T00:34:51.431 INFO:teuthology.orchestra.run.smithi141.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T00:34:51.432 INFO:teuthology.orchestra.run.smithi141.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T00:34:51.432 INFO:teuthology.orchestra.run.smithi141.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T00:34:51.432 INFO:teuthology.orchestra.run.smithi141.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T00:34:51.432 INFO:teuthology.orchestra.run.smithi141.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T00:34:51.433 INFO:teuthology.orchestra.run.smithi141.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-23T00:34:51.433 INFO:teuthology.orchestra.run.smithi141.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-23T00:34:51.433 INFO:teuthology.orchestra.run.smithi141.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T00:34:51.433 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T00:34:51.433 INFO:teuthology.orchestra.run.smithi141.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T00:34:51.434 INFO:teuthology.orchestra.run.smithi141.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T00:34:51.434 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T00:34:51.434 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T00:34:51.434 INFO:teuthology.orchestra.run.smithi141.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T00:34:51.434 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.435 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.435 INFO:teuthology.orchestra.run.smithi141.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T00:34:51.435 INFO:teuthology.orchestra.run.smithi141.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T00:34:51.435 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.435 INFO:teuthology.orchestra.run.smithi141.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T00:34:51.435 INFO:teuthology.orchestra.run.smithi141.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T00:34:51.435 INFO:teuthology.orchestra.run.smithi141.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-23T00:34:51.436 INFO:teuthology.orchestra.run.smithi141.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T00:34:51.436 INFO:teuthology.orchestra.run.smithi141.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T00:34:51.436 INFO:teuthology.orchestra.run.smithi141.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T00:34:51.436 INFO:teuthology.orchestra.run.smithi141.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T00:34:51.436 INFO:teuthology.orchestra.run.smithi141.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T00:34:51.436 INFO:teuthology.orchestra.run.smithi141.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T00:34:51.436 INFO:teuthology.orchestra.run.smithi141.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T00:34:51.437 INFO:teuthology.orchestra.run.smithi141.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T00:34:51.437 INFO:teuthology.orchestra.run.smithi141.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T00:34:51.437 INFO:teuthology.orchestra.run.smithi141.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T00:34:51.437 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.437 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.437 INFO:teuthology.orchestra.run.smithi141.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:51.437 INFO:teuthology.orchestra.run.smithi141.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-23T00:34:51.438 INFO:teuthology.orchestra.run.smithi141.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T00:34:51.438 INFO:teuthology.orchestra.run.smithi141.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-23T00:34:51.438 INFO:teuthology.orchestra.run.smithi141.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T00:34:51.438 INFO:teuthology.orchestra.run.smithi141.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-23T00:34:51.439 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:34:51.439 INFO:teuthology.orchestra.run.smithi141.stdout:Complete! 2023-06-23T00:34:51.616 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 110/122 2023-06-23T00:34:51.756 DEBUG:teuthology.parallel:result is None 2023-06-23T00:34:51.897 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-06-23T00:34:52.234 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-06-23T00:34:52.247 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-06-23T00:34:52.449 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-23T00:34:52.632 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-23T00:34:52.836 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-23T00:34:53.026 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-23T00:34:53.330 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-23T00:34:53.584 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-06-23T00:34:53.649 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-06-23T00:34:53.859 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-06-23T00:34:53.871 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-06-23T00:34:54.004 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-06-23T00:34:54.004 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T00:34:54.246 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T00:34:54.247 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T00:34:54.931 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T00:34:54.932 INFO:teuthology.orchestra.run.smithi055.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T00:34:54.932 INFO:teuthology.orchestra.run.smithi055.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T00:34:54.932 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:34:55.298 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-23T00:34:55.299 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-23T00:34:55.299 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-23T00:34:55.299 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T00:34:55.299 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-23T00:34:55.299 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T00:34:55.299 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T00:34:55.300 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-23T00:34:55.300 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-23T00:34:55.300 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T00:34:55.300 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-23T00:34:55.300 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-23T00:34:55.300 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T00:34:55.300 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-23T00:34:55.301 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-23T00:34:55.301 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-23T00:34:55.301 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-23T00:34:55.301 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-23T00:34:55.301 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-23T00:34:55.301 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-23T00:34:55.301 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-23T00:34:55.301 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-23T00:34:55.302 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-23T00:34:55.302 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-23T00:34:55.302 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-23T00:34:55.302 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-23T00:34:55.302 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-23T00:34:55.302 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T00:34:55.302 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-23T00:34:55.302 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-23T00:34:55.303 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-23T00:34:55.303 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-23T00:34:55.303 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-23T00:34:55.303 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-23T00:34:55.303 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-23T00:34:55.303 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-23T00:34:55.303 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-23T00:34:55.304 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-23T00:34:55.304 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-23T00:34:55.304 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-23T00:34:55.304 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-23T00:34:55.304 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-23T00:34:55.304 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-23T00:34:55.304 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-23T00:34:55.304 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-23T00:34:55.305 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-23T00:34:55.305 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-23T00:34:55.305 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-23T00:34:55.305 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-23T00:34:55.305 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-23T00:34:55.305 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-23T00:34:55.305 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-23T00:34:55.305 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-23T00:34:55.306 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-23T00:34:55.306 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-23T00:34:55.306 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-23T00:34:55.306 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-23T00:34:55.306 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-23T00:34:55.306 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-23T00:34:55.306 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-23T00:34:55.306 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-23T00:34:55.307 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-23T00:34:55.307 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-23T00:34:55.307 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-23T00:34:55.307 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-23T00:34:55.307 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-23T00:34:55.307 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-23T00:34:55.307 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-23T00:34:55.307 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-23T00:34:55.308 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-23T00:34:55.308 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-23T00:34:55.308 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-06-23T00:34:55.308 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-06-23T00:34:55.308 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-06-23T00:34:55.308 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-06-23T00:34:55.308 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-06-23T00:34:55.309 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-06-23T00:34:55.309 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-06-23T00:34:55.309 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-06-23T00:34:55.309 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-06-23T00:34:55.309 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-06-23T00:34:55.309 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-06-23T00:34:55.309 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-06-23T00:34:55.309 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-06-23T00:34:55.310 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-06-23T00:34:55.310 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-06-23T00:34:55.310 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-06-23T00:34:55.310 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-06-23T00:34:55.310 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-06-23T00:34:55.310 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-06-23T00:34:55.310 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-06-23T00:34:55.311 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-06-23T00:34:55.311 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-06-23T00:34:55.311 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-06-23T00:34:55.311 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-06-23T00:34:55.311 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-06-23T00:34:55.311 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-06-23T00:34:55.311 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-06-23T00:34:55.311 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-06-23T00:34:55.312 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-06-23T00:34:55.312 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-06-23T00:34:55.312 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-06-23T00:34:55.312 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-06-23T00:34:55.313 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-06-23T00:34:55.313 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-06-23T00:34:55.313 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-06-23T00:34:55.313 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-06-23T00:34:55.313 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-06-23T00:34:55.313 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-06-23T00:34:55.313 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-06-23T00:34:55.314 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-06-23T00:34:55.314 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-06-23T00:34:55.314 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-06-23T00:34:55.314 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-06-23T00:34:55.314 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-06-23T00:34:55.314 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-06-23T00:34:55.314 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-06-23T00:34:55.314 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-06-23T00:34:55.315 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-23T00:34:55.315 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-23T00:34:55.315 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-23T00:34:57.630 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T00:34:57.631 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:01:49 ago on Fri 23 Jun 2023 12:33:08 AM UTC. 2023-06-23T00:34:59.639 INFO:teuthology.orchestra.run.smithi055.stdout:Installed products updated. 2023-06-23T00:34:59.791 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:34:59.792 INFO:teuthology.orchestra.run.smithi055.stdout:Upgraded: 2023-06-23T00:34:59.792 INFO:teuthology.orchestra.run.smithi055.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.792 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.792 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-06-23T00:34:59.792 INFO:teuthology.orchestra.run.smithi055.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T00:34:59.792 INFO:teuthology.orchestra.run.smithi055.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T00:34:59.793 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.793 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.793 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.793 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.793 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:59.793 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.793 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:59.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:59.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:59.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:59.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:59.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.795 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.795 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:59.795 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.795 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.795 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.795 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:59.796 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:59.796 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.797 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T00:34:59.797 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-23T00:34:59.797 INFO:teuthology.orchestra.run.smithi055.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T00:34:59.797 INFO:teuthology.orchestra.run.smithi055.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-23T00:34:59.798 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T00:34:59.798 INFO:teuthology.orchestra.run.smithi055.stdout: jq-1.5-12.el8.x86_64 2023-06-23T00:34:59.798 INFO:teuthology.orchestra.run.smithi055.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-23T00:34:59.799 INFO:teuthology.orchestra.run.smithi055.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-23T00:34:59.799 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T00:34:59.799 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.800 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.800 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.800 INFO:teuthology.orchestra.run.smithi055.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-23T00:34:59.801 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-23T00:34:59.801 INFO:teuthology.orchestra.run.smithi055.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T00:34:59.801 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-23T00:34:59.802 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T00:34:59.802 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.802 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.803 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-23T00:34:59.803 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.803 INFO:teuthology.orchestra.run.smithi055.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T00:34:59.803 INFO:teuthology.orchestra.run.smithi055.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-23T00:34:59.804 INFO:teuthology.orchestra.run.smithi055.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T00:34:59.805 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T00:34:59.805 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-23T00:34:59.805 INFO:teuthology.orchestra.run.smithi055.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T00:34:59.805 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T00:34:59.806 INFO:teuthology.orchestra.run.smithi055.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T00:34:59.806 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-23T00:34:59.806 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-23T00:34:59.806 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-23T00:34:59.806 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T00:34:59.806 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T00:34:59.807 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T00:34:59.807 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T00:34:59.807 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-23T00:34:59.807 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T00:34:59.807 INFO:teuthology.orchestra.run.smithi055.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T00:34:59.807 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T00:34:59.807 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.808 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.808 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.808 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T00:34:59.808 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T00:34:59.808 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T00:34:59.808 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T00:34:59.809 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T00:34:59.809 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T00:34:59.809 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-23T00:34:59.809 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T00:34:59.809 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T00:34:59.809 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T00:34:59.810 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T00:34:59.810 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T00:34:59.810 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T00:34:59.810 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T00:34:59.810 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T00:34:59.810 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T00:34:59.810 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-23T00:34:59.811 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-23T00:34:59.811 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T00:34:59.811 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T00:34:59.811 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T00:34:59.811 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T00:34:59.811 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T00:34:59.811 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T00:34:59.811 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T00:34:59.812 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.812 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.812 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T00:34:59.812 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T00:34:59.812 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.812 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T00:34:59.813 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T00:34:59.813 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-23T00:34:59.813 INFO:teuthology.orchestra.run.smithi055.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T00:34:59.813 INFO:teuthology.orchestra.run.smithi055.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T00:34:59.813 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T00:34:59.813 INFO:teuthology.orchestra.run.smithi055.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T00:34:59.813 INFO:teuthology.orchestra.run.smithi055.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T00:34:59.814 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T00:34:59.814 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T00:34:59.814 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T00:34:59.815 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T00:34:59.815 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T00:34:59.815 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.815 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.816 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T00:34:59.816 INFO:teuthology.orchestra.run.smithi055.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-23T00:34:59.816 INFO:teuthology.orchestra.run.smithi055.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T00:34:59.816 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-23T00:34:59.816 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T00:34:59.817 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-23T00:34:59.817 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:34:59.817 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-06-23T00:35:00.112 DEBUG:teuthology.parallel:result is None 2023-06-23T00:35:00.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T00:35:00.329 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T00:35:00.352 INFO:teuthology.orchestra.run.smithi055.stdout:17.2.6-719.g42ddc46e.el8 2023-06-23T00:35:00.353 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-23T00:35:00.353 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-23T00:35:00.354 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T00:35:00.570 DEBUG:teuthology.orchestra.run.smithi141:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T00:35:00.594 INFO:teuthology.orchestra.run.smithi141.stdout:17.2.6-719.g42ddc46e.el8 2023-06-23T00:35:00.594 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-23T00:35:00.594 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-23T00:35:00.596 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-23T00:35:00.597 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:00.598 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T00:35:00.641 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:00.642 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T00:35:00.682 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-23T00:35:00.682 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:00.683 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T00:35:00.711 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T00:35:00.778 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:00.778 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T00:35:00.804 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T00:35:00.870 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-23T00:35:00.871 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:00.871 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T00:35:00.898 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T00:35:00.963 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:00.964 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T00:35:00.988 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T00:35:01.053 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-23T00:35:01.053 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:01.053 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T00:35:01.078 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T00:35:01.143 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:01.144 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T00:35:01.169 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T00:35:01.233 INFO:teuthology.run_tasks:Running task ceph... 2023-06-23T00:35:01.318 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-06-23T00:35:01.319 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 777 /var/log/ceph 2023-06-23T00:35:01.322 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod 777 /var/log/ceph 2023-06-23T00:35:01.346 INFO:tasks.ceph:Disabling ceph logrotate... 2023-06-23T00:35:01.347 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-23T00:35:01.388 DEBUG:teuthology.orchestra.run.smithi141:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-23T00:35:01.411 INFO:tasks.ceph:Creating extra log directories... 2023-06-23T00:35:01.412 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-23T00:35:01.453 DEBUG:teuthology.orchestra.run.smithi141:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-23T00:35:01.483 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-06-23T00:35:01.484 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-06-23T00:35:01.484 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-06-23T00:35:01.484 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:01.485 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-23T00:35:01.531 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-23T00:35:01.646 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-23T00:35:01.672 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:01.672 DEBUG:teuthology.orchestra.run.smithi141:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-23T00:35:01.687 DEBUG:teuthology.orchestra.run.smithi141:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-23T00:35:01.801 DEBUG:teuthology.orchestra.run.smithi141:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-23T00:35:01.831 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-06-23T00:35:01.831 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-06-23T00:35:01.831 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882', 'branch': 'wip-yuri4-testing-2023-06-21-0846-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7311882', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2023-06-21-0846-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri4-testing-2023-06-21-0846-quincy', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9037, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri4-testing-2023-06-21-0846-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'targets': {'smithi055.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC75fSi2oSvKw3YNS36qaAND2urjwcW/v9YT2M3JHk9YXUiOrW0/APthQChdXlMU8j5NDTGTGQX6k2gD31Z7zcB7ZsZ0jxUAgZehlukP0Ss64qMcKhiMdbO/sbQeaHykNmMA1xQlfK91IILupQeAGwQihBbq8ZHEZEc/W9MppBndcjV0Wb/e1e69oAsVmjwd/supjKa6sIEQiCUSuD02zFhfdQIgb+j8GB4mGiQuSwBDT8bhOAtSnaQTXJCKWymD/FdqmSDIUA/dLRdRtxxFg2/u0ANS9350Y/0KWYcgcIPFvCdCAPtW5knw4XhZVnADAWnZ47cV/9MIiTJqYey0zp+t7W8K1Noxa3VkgUdnyKTcD7LTR4XHGFZ1dk4q3LYRV+4cJrjIVtIdC8+3YXaNBL2z0MRhEWtbIQ0vWtiJmOTgrfSKEiGw1blxYOgCiePEKjv956ggIEbQEeq4/DXdq0mJObpfpFc/jKwshNWscFv+gUuGfiEEJpp5kdu0IOdd98=', 'smithi141.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIXpQjeegzj7I7Qy7Nhs98P522TO+rOlW/zrMBOYH1mMrcIPOJrEA61LILrT0RBuJUpdgtAZhwazYthRMQN5gohIHl4hQy/ahqG+m5nDAP2hZY5sGwad0HgEZB93bjdhPNRBLLKHHpRIIMXZImUsNNwdMpuQxeDtv2/Ts72Qkk/RZ5FpEUkGBIjJ5nfZVtFBF2Sjl1jrZ7eiuoKotuhY1UrO+wcR47RCAQ96xXChs2/h9+vZoag3hcqNGf5oSVZV4QpdWOGmlWPhPrdfkeitw9qVC0UnrKdwvYwuVoO9KdQQYT7JuQIHSei2vw18UeEqubajpjsf+/HePOhzDcNcO5LQc2UU4ouKrntIYH02/3TLUtIFLrvmNXrK6036O/Pg7mVFa9wZ11nelPmbeWCcHME3/8Kki/WyaRKYXZXw+uCCnG/OuCSliYYK49Ap1UvZuku3Qy3jObqADWbT0CMwfxeSX6enw3yPb4hbXmIoC3cd8tt9TMpakPr3x525uNkXc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}}, {'ceph': None}, {'kclient': None}, {'check-counter': {'counters': {'mgr': [{'min': 4, 'name': 'finisher-volumes.complete_latency.avgcount'}, {'expected_val': 0, 'name': 'finisher-volumes.queue_len'}]}}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestPerModuleFinsherThread']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '076bbebc42a14f7d568aaa78eabb0038327bcb23', 'timestamp': '2023-06-22_15:25:25', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-06-23T00:35:01.832 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-23T00:35:01.835 DEBUG:teuthology.orchestra.run.smithi141:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-23T00:35:01.880 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m0777 -- /var/run/ceph 2023-06-23T00:35:01.889 DEBUG:teuthology.orchestra.run.smithi141:> sudo install -d -m0777 -- /var/run/ceph 2023-06-23T00:35:01.945 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:01.945 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T00:35:01.968 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T00:35:01.968 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_1 2023-06-23T00:35:02.023 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T00:35:02.023 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T00:35:02.023 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 208625 Links: 1 2023-06-23T00:35:02.024 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T00:35:02.024 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T00:35:02.024 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2023-06-23 00:34:54.840924195 +0000 2023-06-23T00:35:02.024 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2023-06-23 00:32:19.801620744 +0000 2023-06-23T00:35:02.024 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2023-06-23 00:32:19.801620744 +0000 2023-06-23T00:35:02.024 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2023-06-23T00:35:02.025 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T00:35:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2023-06-23T00:35:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2023-06-23T00:35:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000324015 s, 1.6 MB/s 2023-06-23T00:35:02.089 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T00:35:02.145 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_2 2023-06-23T00:35:02.201 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T00:35:02.201 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T00:35:02.201 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 210572 Links: 1 2023-06-23T00:35:02.201 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T00:35:02.201 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T00:35:02.202 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2023-06-23 00:34:54.840924195 +0000 2023-06-23T00:35:02.202 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2023-06-23 00:32:20.362609200 +0000 2023-06-23T00:35:02.202 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2023-06-23 00:32:20.362609200 +0000 2023-06-23T00:35:02.202 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2023-06-23T00:35:02.202 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T00:35:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2023-06-23T00:35:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2023-06-23T00:35:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000320454 s, 1.6 MB/s 2023-06-23T00:35:02.267 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T00:35:02.323 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_3 2023-06-23T00:35:02.378 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T00:35:02.378 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T00:35:02.379 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 213123 Links: 1 2023-06-23T00:35:02.379 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T00:35:02.379 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T00:35:02.379 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2023-06-23 00:34:54.840924195 +0000 2023-06-23T00:35:02.379 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2023-06-23 00:32:20.911597903 +0000 2023-06-23T00:35:02.379 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2023-06-23 00:32:20.911597903 +0000 2023-06-23T00:35:02.379 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2023-06-23T00:35:02.380 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T00:35:02.444 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2023-06-23T00:35:02.445 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2023-06-23T00:35:02.445 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000302843 s, 1.7 MB/s 2023-06-23T00:35:02.445 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T00:35:02.500 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_4 2023-06-23T00:35:02.555 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T00:35:02.556 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T00:35:02.556 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 215256 Links: 1 2023-06-23T00:35:02.556 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T00:35:02.557 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T00:35:02.557 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2023-06-23 00:34:54.841924175 +0000 2023-06-23T00:35:02.557 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2023-06-23 00:32:21.438587058 +0000 2023-06-23T00:35:02.557 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2023-06-23 00:32:21.438587058 +0000 2023-06-23T00:35:02.557 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2023-06-23T00:35:02.557 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T00:35:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2023-06-23T00:35:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2023-06-23T00:35:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000298635 s, 1.7 MB/s 2023-06-23T00:35:02.621 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T00:35:02.677 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T00:35:02.678 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:02.678 DEBUG:teuthology.orchestra.run.smithi141:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T00:35:02.694 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T00:35:02.695 DEBUG:teuthology.orchestra.run.smithi141:> stat /dev/vg_nvme/lv_1 2023-06-23T00:35:02.751 INFO:teuthology.orchestra.run.smithi141.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T00:35:02.752 INFO:teuthology.orchestra.run.smithi141.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T00:35:02.752 INFO:teuthology.orchestra.run.smithi141.stdout:Device: 6h/6d Inode: 213573 Links: 1 2023-06-23T00:35:02.753 INFO:teuthology.orchestra.run.smithi141.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T00:35:02.753 INFO:teuthology.orchestra.run.smithi141.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T00:35:02.753 INFO:teuthology.orchestra.run.smithi141.stdout:Access: 2023-06-23 00:34:46.427239231 +0000 2023-06-23T00:35:02.753 INFO:teuthology.orchestra.run.smithi141.stdout:Modify: 2023-06-23 00:32:07.103277692 +0000 2023-06-23T00:35:02.754 INFO:teuthology.orchestra.run.smithi141.stdout:Change: 2023-06-23 00:32:07.103277692 +0000 2023-06-23T00:35:02.754 INFO:teuthology.orchestra.run.smithi141.stdout: Birth: - 2023-06-23T00:35:02.754 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T00:35:02.815 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records in 2023-06-23T00:35:02.816 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records out 2023-06-23T00:35:02.817 INFO:teuthology.orchestra.run.smithi141.stderr:512 bytes copied, 0.000263166 s, 1.9 MB/s 2023-06-23T00:35:02.819 DEBUG:teuthology.orchestra.run.smithi141:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T00:35:02.875 DEBUG:teuthology.orchestra.run.smithi141:> stat /dev/vg_nvme/lv_2 2023-06-23T00:35:02.930 INFO:teuthology.orchestra.run.smithi141.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T00:35:02.930 INFO:teuthology.orchestra.run.smithi141.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T00:35:02.930 INFO:teuthology.orchestra.run.smithi141.stdout:Device: 6h/6d Inode: 213649 Links: 1 2023-06-23T00:35:02.930 INFO:teuthology.orchestra.run.smithi141.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T00:35:02.930 INFO:teuthology.orchestra.run.smithi141.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T00:35:02.930 INFO:teuthology.orchestra.run.smithi141.stdout:Access: 2023-06-23 00:34:46.427239231 +0000 2023-06-23T00:35:02.931 INFO:teuthology.orchestra.run.smithi141.stdout:Modify: 2023-06-23 00:32:07.644266479 +0000 2023-06-23T00:35:02.931 INFO:teuthology.orchestra.run.smithi141.stdout:Change: 2023-06-23 00:32:07.644266479 +0000 2023-06-23T00:35:02.931 INFO:teuthology.orchestra.run.smithi141.stdout: Birth: - 2023-06-23T00:35:02.931 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T00:35:02.994 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records in 2023-06-23T00:35:02.994 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records out 2023-06-23T00:35:02.995 INFO:teuthology.orchestra.run.smithi141.stderr:512 bytes copied, 0.000224384 s, 2.3 MB/s 2023-06-23T00:35:02.995 DEBUG:teuthology.orchestra.run.smithi141:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T00:35:03.053 DEBUG:teuthology.orchestra.run.smithi141:> stat /dev/vg_nvme/lv_3 2023-06-23T00:35:03.108 INFO:teuthology.orchestra.run.smithi141.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T00:35:03.108 INFO:teuthology.orchestra.run.smithi141.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T00:35:03.109 INFO:teuthology.orchestra.run.smithi141.stdout:Device: 6h/6d Inode: 211681 Links: 1 2023-06-23T00:35:03.109 INFO:teuthology.orchestra.run.smithi141.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T00:35:03.109 INFO:teuthology.orchestra.run.smithi141.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T00:35:03.109 INFO:teuthology.orchestra.run.smithi141.stdout:Access: 2023-06-23 00:34:46.427239231 +0000 2023-06-23T00:35:03.110 INFO:teuthology.orchestra.run.smithi141.stdout:Modify: 2023-06-23 00:32:08.162255742 +0000 2023-06-23T00:35:03.110 INFO:teuthology.orchestra.run.smithi141.stdout:Change: 2023-06-23 00:32:08.162255742 +0000 2023-06-23T00:35:03.110 INFO:teuthology.orchestra.run.smithi141.stdout: Birth: - 2023-06-23T00:35:03.111 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T00:35:03.170 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records in 2023-06-23T00:35:03.171 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records out 2023-06-23T00:35:03.171 INFO:teuthology.orchestra.run.smithi141.stderr:512 bytes copied, 0.000237814 s, 2.2 MB/s 2023-06-23T00:35:03.172 DEBUG:teuthology.orchestra.run.smithi141:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T00:35:03.227 DEBUG:teuthology.orchestra.run.smithi141:> stat /dev/vg_nvme/lv_4 2023-06-23T00:35:03.282 INFO:teuthology.orchestra.run.smithi141.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T00:35:03.283 INFO:teuthology.orchestra.run.smithi141.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T00:35:03.283 INFO:teuthology.orchestra.run.smithi141.stdout:Device: 6h/6d Inode: 210692 Links: 1 2023-06-23T00:35:03.283 INFO:teuthology.orchestra.run.smithi141.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T00:35:03.284 INFO:teuthology.orchestra.run.smithi141.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T00:35:03.285 INFO:teuthology.orchestra.run.smithi141.stdout:Access: 2023-06-23 00:34:46.427239231 +0000 2023-06-23T00:35:03.285 INFO:teuthology.orchestra.run.smithi141.stdout:Modify: 2023-06-23 00:32:08.680245006 +0000 2023-06-23T00:35:03.285 INFO:teuthology.orchestra.run.smithi141.stdout:Change: 2023-06-23 00:32:08.680245006 +0000 2023-06-23T00:35:03.285 INFO:teuthology.orchestra.run.smithi141.stdout: Birth: - 2023-06-23T00:35:03.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T00:35:03.345 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records in 2023-06-23T00:35:03.345 INFO:teuthology.orchestra.run.smithi141.stderr:1+0 records out 2023-06-23T00:35:03.346 INFO:teuthology.orchestra.run.smithi141.stderr:512 bytes copied, 0.000287699 s, 1.8 MB/s 2023-06-23T00:35:03.349 DEBUG:teuthology.orchestra.run.smithi141:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T00:35:03.404 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T00:35:03.404 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi055.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi141.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-23T00:35:03.404 INFO:tasks.ceph:Generating config... 2023-06-23T00:35:03.406 INFO:tasks.ceph:[client] client mount timeout = 600 2023-06-23T00:35:03.406 INFO:tasks.ceph:[client] debug client = 20 2023-06-23T00:35:03.406 INFO:tasks.ceph:[client] debug ms = 1 2023-06-23T00:35:03.406 INFO:tasks.ceph:[client] pid file = 2023-06-23T00:35:03.407 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-06-23T00:35:03.407 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-06-23T00:35:03.407 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-06-23T00:35:03.407 INFO:tasks.ceph:[mds] debug mds = 20 2023-06-23T00:35:03.407 INFO:tasks.ceph:[mds] debug ms = 1 2023-06-23T00:35:03.408 INFO:tasks.ceph:[mds] mds debug frag = True 2023-06-23T00:35:03.408 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-06-23T00:35:03.408 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-06-23T00:35:03.408 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-06-23T00:35:03.408 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-06-23T00:35:03.409 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-06-23T00:35:03.409 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-06-23T00:35:03.409 INFO:tasks.ceph:[mgr] debug client = 20 2023-06-23T00:35:03.409 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-06-23T00:35:03.409 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-06-23T00:35:03.409 INFO:tasks.ceph:[mgr] debug ms = 1 2023-06-23T00:35:03.410 INFO:tasks.ceph:[mon] debug mon = 20 2023-06-23T00:35:03.410 INFO:tasks.ceph:[mon] debug ms = 1 2023-06-23T00:35:03.410 INFO:tasks.ceph:[mon] debug paxos = 20 2023-06-23T00:35:03.410 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-06-23T00:35:03.410 INFO:tasks.ceph:[osd] bdev async discard = True 2023-06-23T00:35:03.411 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-06-23T00:35:03.411 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-06-23T00:35:03.411 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-06-23T00:35:03.411 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-06-23T00:35:03.411 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-06-23T00:35:03.412 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-06-23T00:35:03.412 INFO:tasks.ceph:[osd] debug ms = 1 2023-06-23T00:35:03.412 INFO:tasks.ceph:[osd] debug osd = 20 2023-06-23T00:35:03.412 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-06-23T00:35:03.412 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-06-23T00:35:03.413 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-06-23T00:35:03.413 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-06-23T00:35:03.413 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-06-23T00:35:03.413 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-06-23T00:35:03.413 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-06-23T00:35:03.414 INFO:tasks.ceph:Setting up mon.a... 2023-06-23T00:35:03.414 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-06-23T00:35:03.484 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.keyring 2023-06-23T00:35:03.486 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-06-23T00:35:03.527 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T00:35:03.593 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.55'), ('mon.b', '172.21.15.141'), ('mon.c', '[v2:172.21.15.141:3301,v1:172.21.15.141:6790]')] 2023-06-23T00:35:03.594 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.55,172.21.15.141,[v2:172.21.15.141:3301,v1:172.21.15.141:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-06-23T00:35:03.596 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:03.596 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-06-23T00:35:03.649 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.55 --add b 172.21.15.141 --addv c '[v2:172.21.15.141:3301,v1:172.21.15.141:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:03.733 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:03.734 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: generated fsid c0a83a0b-dd71-486d-bdc4-a203f7ef404f 2023-06-23T00:35:03.734 INFO:teuthology.orchestra.run.smithi055.stdout:setting min_mon_release = octopus 2023-06-23T00:35:03.734 INFO:teuthology.orchestra.run.smithi055.stdout:epoch 0 2023-06-23T00:35:03.735 INFO:teuthology.orchestra.run.smithi055.stdout:fsid c0a83a0b-dd71-486d-bdc4-a203f7ef404f 2023-06-23T00:35:03.735 INFO:teuthology.orchestra.run.smithi055.stdout:last_changed 2023-06-23T00:35:03.731539+0000 2023-06-23T00:35:03.735 INFO:teuthology.orchestra.run.smithi055.stdout:created 2023-06-23T00:35:03.731539+0000 2023-06-23T00:35:03.735 INFO:teuthology.orchestra.run.smithi055.stdout:min_mon_release 15 (octopus) 2023-06-23T00:35:03.736 INFO:teuthology.orchestra.run.smithi055.stdout:election_strategy: 1 2023-06-23T00:35:03.736 INFO:teuthology.orchestra.run.smithi055.stdout:0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2023-06-23T00:35:03.736 INFO:teuthology.orchestra.run.smithi055.stdout:1: [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] mon.b 2023-06-23T00:35:03.737 INFO:teuthology.orchestra.run.smithi055.stdout:2: [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] mon.c 2023-06-23T00:35:03.737 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-06-23T00:35:03.738 DEBUG:teuthology.orchestra.run.smithi055:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-06-23T00:35:03.793 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c0a83a0b-dd71-486d-bdc4-a203f7ef404f... 2023-06-23T00:35:03.795 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-23T00:35:03.835 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-23T00:35:03.877 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2023-06-23T00:35:03.877 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2023-06-23T00:35:03.878 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-23T00:35:03.878 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2023-06-23T00:35:03.878 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.878 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2023-06-23T00:35:03.878 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.878 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2023-06-23T00:35:03.879 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.879 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2023-06-23T00:35:03.879 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2023-06-23T00:35:03.879 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.879 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2023-06-23T00:35:03.879 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2023-06-23T00:35:03.879 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.879 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-23T00:35:03.880 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2023-06-23T00:35:03.880 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.880 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2023-06-23T00:35:03.880 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-23T00:35:03.880 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.880 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2023-06-23T00:35:03.880 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.881 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2023-06-23T00:35:03.881 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2023-06-23T00:35:03.881 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2023-06-23T00:35:03.881 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2023-06-23T00:35:03.881 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2023-06-23T00:35:03.881 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2023-06-23T00:35:03.881 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2023-06-23T00:35:03.881 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-23T00:35:03.882 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-23T00:35:03.882 INFO:teuthology.orchestra.run.smithi055.stdout: mon_allow_pool_size_one = true 2023-06-23T00:35:03.882 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.882 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-23T00:35:03.882 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.882 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2023-06-23T00:35:03.882 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.883 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2023-06-23T00:35:03.883 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.883 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2023-06-23T00:35:03.883 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2023-06-23T00:35:03.883 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2023-06-23T00:35:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.55,172.21.15.141,[v2:172.21.15.141:3301,v1:172.21.15.141:6790]" 2023-06-23T00:35:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on skipped message = False 2023-06-23T00:35:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = c0a83a0b-dd71-486d-bdc4-a203f7ef404f 2023-06-23T00:35:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.884 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2023-06-23T00:35:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2023-06-23T00:35:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2023-06-23T00:35:03.885 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2023-06-23T00:35:03.885 INFO:teuthology.orchestra.run.smithi055.stdout: osd mclock profile = high_recovery_ops 2023-06-23T00:35:03.885 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.885 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2023-06-23T00:35:03.885 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2023-06-23T00:35:03.885 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.885 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2023-06-23T00:35:03.885 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2023-06-23T00:35:03.886 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2023-06-23T00:35:03.886 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.886 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2023-06-23T00:35:03.886 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2023-06-23T00:35:03.886 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.886 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2023-06-23T00:35:03.886 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.886 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2023-06-23T00:35:03.887 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.887 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2023-06-23T00:35:03.888 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.888 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2023-06-23T00:35:03.888 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2023-06-23T00:35:03.888 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.888 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2023-06-23T00:35:03.888 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2023-06-23T00:35:03.888 INFO:teuthology.orchestra.run.smithi055.stdout: bdev async discard = True 2023-06-23T00:35:03.889 INFO:teuthology.orchestra.run.smithi055.stdout: bdev enable discard = True 2023-06-23T00:35:03.889 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore allocator = bitmap 2023-06-23T00:35:03.889 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2023-06-23T00:35:03.889 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2023-06-23T00:35:03.890 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 1/20 2023-06-23T00:35:03.890 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 1/20 2023-06-23T00:35:03.890 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2023-06-23T00:35:03.890 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2023-06-23T00:35:03.890 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 4/10 2023-06-23T00:35:03.890 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2023-06-23T00:35:03.891 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2023-06-23T00:35:03.891 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2023-06-23T00:35:03.891 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2023-06-23T00:35:03.891 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2023-06-23T00:35:03.891 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2023-06-23T00:35:03.891 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.891 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2023-06-23T00:35:03.892 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2023-06-23T00:35:03.892 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2023-06-23T00:35:03.892 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2023-06-23T00:35:03.892 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2023-06-23T00:35:03.892 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2023-06-23T00:35:03.892 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2023-06-23T00:35:03.892 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2023-06-23T00:35:03.893 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2023-06-23T00:35:03.893 INFO:teuthology.orchestra.run.smithi055.stdout: debug finisher = 20 2023-06-23T00:35:03.893 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.893 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2023-06-23T00:35:03.893 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2023-06-23T00:35:03.893 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2023-06-23T00:35:03.894 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2023-06-23T00:35:03.894 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2023-06-23T00:35:03.894 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2023-06-23T00:35:03.894 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2023-06-23T00:35:03.894 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2023-06-23T00:35:03.895 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2023-06-23T00:35:03.895 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2023-06-23T00:35:03.895 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2023-06-23T00:35:03.895 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.896 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-23T00:35:03.896 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2023-06-23T00:35:03.896 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2023-06-23T00:35:03.896 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.897 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2023-06-23T00:35:03.897 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-23T00:35:03.897 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-23T00:35:03.897 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.898 INFO:teuthology.orchestra.run.smithi055.stdout: # 1m isn't quite enough 2023-06-23T00:35:03.898 INFO:teuthology.orchestra.run.smithi055.stdout: mon_down_mkfs_grace = 2m 2023-06-23T00:35:03.898 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.898 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_filestore_osds = false 2023-06-23T00:35:03.899 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2023-06-23T00:35:03.899 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:03.899 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2023-06-23T00:35:03.900 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2023-06-23T00:35:03.900 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2023-06-23T00:35:03.900 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2023-06-23T00:35:03.901 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-23T00:35:03.901 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-23T00:35:03.901 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2023-06-23T00:35:03.901 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2023-06-23T00:35:03.902 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2023-06-23T00:35:03.902 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = "" 2023-06-23T00:35:03.902 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2023-06-23T00:35:03.903 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2023-06-23T00:35:03.903 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2023-06-23T00:35:03.903 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2023-06-23T00:35:03.903 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2023-06-23T00:35:03.903 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2023-06-23T00:35:03.904 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2023-06-23T00:35:03.904 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2023-06-23T00:35:03.904 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2023-06-23T00:35:03.904 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2023-06-23T00:35:03.905 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2023-06-23T00:35:03.905 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2023-06-23T00:35:03.905 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2023-06-23T00:35:03.905 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2023-06-23T00:35:03.906 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2023-06-23T00:35:03.906 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2023-06-23T00:35:03.906 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2023-06-23T00:35:03.906 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2023-06-23T00:35:03.907 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2023-06-23T00:35:03.908 INFO:teuthology.orchestra.run.smithi141.stdout:[global] 2023-06-23T00:35:03.908 INFO:teuthology.orchestra.run.smithi141.stdout: chdir = "" 2023-06-23T00:35:03.909 INFO:teuthology.orchestra.run.smithi141.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-23T00:35:03.909 INFO:teuthology.orchestra.run.smithi141.stdout: auth supported = cephx 2023-06-23T00:35:03.909 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.909 INFO:teuthology.orchestra.run.smithi141.stdout: filestore xattr use omap = true 2023-06-23T00:35:03.909 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.909 INFO:teuthology.orchestra.run.smithi141.stdout: mon clock drift allowed = 1.000 2023-06-23T00:35:03.909 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.909 INFO:teuthology.orchestra.run.smithi141.stdout: osd crush chooseleaf type = 0 2023-06-23T00:35:03.910 INFO:teuthology.orchestra.run.smithi141.stdout: auth debug = true 2023-06-23T00:35:03.910 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.910 INFO:teuthology.orchestra.run.smithi141.stdout: ms die on old message = true 2023-06-23T00:35:03.910 INFO:teuthology.orchestra.run.smithi141.stdout: ms die on bug = true 2023-06-23T00:35:03.910 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.910 INFO:teuthology.orchestra.run.smithi141.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-23T00:35:03.910 INFO:teuthology.orchestra.run.smithi141.stdout: mon pg warn max object skew = 0 2023-06-23T00:35:03.911 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.911 INFO:teuthology.orchestra.run.smithi141.stdout: # disable pg_autoscaler by default for new pools 2023-06-23T00:35:03.911 INFO:teuthology.orchestra.run.smithi141.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-23T00:35:03.912 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.912 INFO:teuthology.orchestra.run.smithi141.stdout: osd pool default size = 2 2023-06-23T00:35:03.912 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.912 INFO:teuthology.orchestra.run.smithi141.stdout: mon osd allow primary affinity = true 2023-06-23T00:35:03.913 INFO:teuthology.orchestra.run.smithi141.stdout: mon osd allow pg remap = true 2023-06-23T00:35:03.913 INFO:teuthology.orchestra.run.smithi141.stdout: mon warn on legacy crush tunables = false 2023-06-23T00:35:03.913 INFO:teuthology.orchestra.run.smithi141.stdout: mon warn on crush straw calc version zero = false 2023-06-23T00:35:03.913 INFO:teuthology.orchestra.run.smithi141.stdout: mon warn on no sortbitwise = false 2023-06-23T00:35:03.914 INFO:teuthology.orchestra.run.smithi141.stdout: mon warn on osd down out interval zero = false 2023-06-23T00:35:03.914 INFO:teuthology.orchestra.run.smithi141.stdout: mon warn on too few osds = false 2023-06-23T00:35:03.914 INFO:teuthology.orchestra.run.smithi141.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-23T00:35:03.915 INFO:teuthology.orchestra.run.smithi141.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-23T00:35:03.915 INFO:teuthology.orchestra.run.smithi141.stdout: mon_allow_pool_size_one = true 2023-06-23T00:35:03.915 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.916 INFO:teuthology.orchestra.run.smithi141.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-23T00:35:03.916 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.916 INFO:teuthology.orchestra.run.smithi141.stdout: osd default data pool replay window = 5 2023-06-23T00:35:03.916 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.916 INFO:teuthology.orchestra.run.smithi141.stdout: mon allow pool delete = true 2023-06-23T00:35:03.917 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.917 INFO:teuthology.orchestra.run.smithi141.stdout: mon cluster log file level = debug 2023-06-23T00:35:03.917 INFO:teuthology.orchestra.run.smithi141.stdout: debug asserts on shutdown = true 2023-06-23T00:35:03.917 INFO:teuthology.orchestra.run.smithi141.stdout: mon health detail to clog = false 2023-06-23T00:35:03.917 INFO:teuthology.orchestra.run.smithi141.stdout: mon host = "172.21.15.55,172.21.15.141,[v2:172.21.15.141:3301,v1:172.21.15.141:6790]" 2023-06-23T00:35:03.917 INFO:teuthology.orchestra.run.smithi141.stdout: ms die on skipped message = False 2023-06-23T00:35:03.918 INFO:teuthology.orchestra.run.smithi141.stdout: fsid = c0a83a0b-dd71-486d-bdc4-a203f7ef404f 2023-06-23T00:35:03.918 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.918 INFO:teuthology.orchestra.run.smithi141.stdout:[osd] 2023-06-23T00:35:03.918 INFO:teuthology.orchestra.run.smithi141.stdout: osd journal size = 100 2023-06-23T00:35:03.918 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.918 INFO:teuthology.orchestra.run.smithi141.stdout: osd scrub load threshold = 5.0 2023-06-23T00:35:03.918 INFO:teuthology.orchestra.run.smithi141.stdout: osd scrub max interval = 600 2023-06-23T00:35:03.919 INFO:teuthology.orchestra.run.smithi141.stdout: osd mclock profile = high_recovery_ops 2023-06-23T00:35:03.919 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.919 INFO:teuthology.orchestra.run.smithi141.stdout: osd recover clone overlap = true 2023-06-23T00:35:03.919 INFO:teuthology.orchestra.run.smithi141.stdout: osd recovery max chunk = 1048576 2023-06-23T00:35:03.919 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.919 INFO:teuthology.orchestra.run.smithi141.stdout: osd debug shutdown = true 2023-06-23T00:35:03.919 INFO:teuthology.orchestra.run.smithi141.stdout: osd debug op order = true 2023-06-23T00:35:03.919 INFO:teuthology.orchestra.run.smithi141.stdout: osd debug verify stray on activate = true 2023-06-23T00:35:03.920 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.920 INFO:teuthology.orchestra.run.smithi141.stdout: osd open classes on start = true 2023-06-23T00:35:03.920 INFO:teuthology.orchestra.run.smithi141.stdout: osd debug pg log writeout = true 2023-06-23T00:35:03.920 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.920 INFO:teuthology.orchestra.run.smithi141.stdout: osd deep scrub update digest min age = 30 2023-06-23T00:35:03.920 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.920 INFO:teuthology.orchestra.run.smithi141.stdout: osd map max advance = 10 2023-06-23T00:35:03.921 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.921 INFO:teuthology.orchestra.run.smithi141.stdout: journal zero on create = true 2023-06-23T00:35:03.921 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.921 INFO:teuthology.orchestra.run.smithi141.stdout: filestore ondisk finisher threads = 3 2023-06-23T00:35:03.921 INFO:teuthology.orchestra.run.smithi141.stdout: filestore apply finisher threads = 3 2023-06-23T00:35:03.921 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.921 INFO:teuthology.orchestra.run.smithi141.stdout: bdev debug aio = true 2023-06-23T00:35:03.921 INFO:teuthology.orchestra.run.smithi141.stdout: osd debug misdirected ops = true 2023-06-23T00:35:03.922 INFO:teuthology.orchestra.run.smithi141.stdout: bdev async discard = True 2023-06-23T00:35:03.922 INFO:teuthology.orchestra.run.smithi141.stdout: bdev enable discard = True 2023-06-23T00:35:03.922 INFO:teuthology.orchestra.run.smithi141.stdout: bluestore allocator = bitmap 2023-06-23T00:35:03.922 INFO:teuthology.orchestra.run.smithi141.stdout: bluestore block size = 96636764160 2023-06-23T00:35:03.922 INFO:teuthology.orchestra.run.smithi141.stdout: bluestore fsck on mount = True 2023-06-23T00:35:03.922 INFO:teuthology.orchestra.run.smithi141.stdout: debug bluefs = 1/20 2023-06-23T00:35:03.922 INFO:teuthology.orchestra.run.smithi141.stdout: debug bluestore = 1/20 2023-06-23T00:35:03.923 INFO:teuthology.orchestra.run.smithi141.stdout: debug ms = 1 2023-06-23T00:35:03.923 INFO:teuthology.orchestra.run.smithi141.stdout: debug osd = 20 2023-06-23T00:35:03.923 INFO:teuthology.orchestra.run.smithi141.stdout: debug rocksdb = 4/10 2023-06-23T00:35:03.923 INFO:teuthology.orchestra.run.smithi141.stdout: mon osd backfillfull_ratio = 0.85 2023-06-23T00:35:03.923 INFO:teuthology.orchestra.run.smithi141.stdout: mon osd full ratio = 0.9 2023-06-23T00:35:03.923 INFO:teuthology.orchestra.run.smithi141.stdout: mon osd nearfull ratio = 0.8 2023-06-23T00:35:03.923 INFO:teuthology.orchestra.run.smithi141.stdout: osd failsafe full ratio = 0.95 2023-06-23T00:35:03.923 INFO:teuthology.orchestra.run.smithi141.stdout: osd objectstore = bluestore 2023-06-23T00:35:03.924 INFO:teuthology.orchestra.run.smithi141.stdout: osd op complaint time = 180 2023-06-23T00:35:03.924 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.924 INFO:teuthology.orchestra.run.smithi141.stdout:[mgr] 2023-06-23T00:35:03.924 INFO:teuthology.orchestra.run.smithi141.stdout: debug ms = 1 2023-06-23T00:35:03.924 INFO:teuthology.orchestra.run.smithi141.stdout: debug mgr = 20 2023-06-23T00:35:03.924 INFO:teuthology.orchestra.run.smithi141.stdout: debug mon = 20 2023-06-23T00:35:03.925 INFO:teuthology.orchestra.run.smithi141.stdout: debug auth = 20 2023-06-23T00:35:03.925 INFO:teuthology.orchestra.run.smithi141.stdout: mon reweight min pgs per osd = 4 2023-06-23T00:35:03.925 INFO:teuthology.orchestra.run.smithi141.stdout: mon reweight min bytes per osd = 10 2023-06-23T00:35:03.925 INFO:teuthology.orchestra.run.smithi141.stdout: mgr/telemetry/nag = false 2023-06-23T00:35:03.925 INFO:teuthology.orchestra.run.smithi141.stdout: debug client = 20 2023-06-23T00:35:03.926 INFO:teuthology.orchestra.run.smithi141.stdout: debug finisher = 20 2023-06-23T00:35:03.926 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.926 INFO:teuthology.orchestra.run.smithi141.stdout:[mon] 2023-06-23T00:35:03.926 INFO:teuthology.orchestra.run.smithi141.stdout: debug ms = 1 2023-06-23T00:35:03.926 INFO:teuthology.orchestra.run.smithi141.stdout: debug mon = 20 2023-06-23T00:35:03.926 INFO:teuthology.orchestra.run.smithi141.stdout: debug paxos = 20 2023-06-23T00:35:03.926 INFO:teuthology.orchestra.run.smithi141.stdout: debug auth = 20 2023-06-23T00:35:03.927 INFO:teuthology.orchestra.run.smithi141.stdout: mon data avail warn = 5 2023-06-23T00:35:03.927 INFO:teuthology.orchestra.run.smithi141.stdout: mon mgr mkfs grace = 240 2023-06-23T00:35:03.927 INFO:teuthology.orchestra.run.smithi141.stdout: mon reweight min pgs per osd = 4 2023-06-23T00:35:03.927 INFO:teuthology.orchestra.run.smithi141.stdout: mon osd reporter subtree level = osd 2023-06-23T00:35:03.927 INFO:teuthology.orchestra.run.smithi141.stdout: mon osd prime pg temp = true 2023-06-23T00:35:03.927 INFO:teuthology.orchestra.run.smithi141.stdout: mon reweight min bytes per osd = 10 2023-06-23T00:35:03.927 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.927 INFO:teuthology.orchestra.run.smithi141.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-23T00:35:03.928 INFO:teuthology.orchestra.run.smithi141.stdout: auth mon ticket ttl = 660 # 11m 2023-06-23T00:35:03.928 INFO:teuthology.orchestra.run.smithi141.stdout: auth service ticket ttl = 240 # 4m 2023-06-23T00:35:03.928 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.928 INFO:teuthology.orchestra.run.smithi141.stdout: # don't complain about insecure global_id in the test suite 2023-06-23T00:35:03.928 INFO:teuthology.orchestra.run.smithi141.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-23T00:35:03.928 INFO:teuthology.orchestra.run.smithi141.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-23T00:35:03.929 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.929 INFO:teuthology.orchestra.run.smithi141.stdout: # 1m isn't quite enough 2023-06-23T00:35:03.929 INFO:teuthology.orchestra.run.smithi141.stdout: mon_down_mkfs_grace = 2m 2023-06-23T00:35:03.929 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.929 INFO:teuthology.orchestra.run.smithi141.stdout: mon_warn_on_filestore_osds = false 2023-06-23T00:35:03.929 INFO:teuthology.orchestra.run.smithi141.stdout: mon op complaint time = 120 2023-06-23T00:35:03.929 INFO:teuthology.orchestra.run.smithi141.stdout: 2023-06-23T00:35:03.929 INFO:teuthology.orchestra.run.smithi141.stdout:[client] 2023-06-23T00:35:03.930 INFO:teuthology.orchestra.run.smithi141.stdout: rgw cache enabled = true 2023-06-23T00:35:03.930 INFO:teuthology.orchestra.run.smithi141.stdout: rgw enable ops log = true 2023-06-23T00:35:03.930 INFO:teuthology.orchestra.run.smithi141.stdout: rgw enable usage log = true 2023-06-23T00:35:03.930 INFO:teuthology.orchestra.run.smithi141.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-23T00:35:03.930 INFO:teuthology.orchestra.run.smithi141.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-23T00:35:03.930 INFO:teuthology.orchestra.run.smithi141.stdout: client mount timeout = 600 2023-06-23T00:35:03.930 INFO:teuthology.orchestra.run.smithi141.stdout: debug client = 20 2023-06-23T00:35:03.931 INFO:teuthology.orchestra.run.smithi141.stdout: debug ms = 1 2023-06-23T00:35:03.931 INFO:teuthology.orchestra.run.smithi141.stdout: pid file = "" 2023-06-23T00:35:03.931 INFO:teuthology.orchestra.run.smithi141.stdout: rados mon op timeout = 900 2023-06-23T00:35:03.931 INFO:teuthology.orchestra.run.smithi141.stdout: rados osd op timeout = 900 2023-06-23T00:35:03.931 INFO:teuthology.orchestra.run.smithi141.stdout:[mon.a] 2023-06-23T00:35:03.931 INFO:teuthology.orchestra.run.smithi141.stdout:[mon.b] 2023-06-23T00:35:03.932 INFO:teuthology.orchestra.run.smithi141.stdout:[mon.c] 2023-06-23T00:35:03.932 INFO:teuthology.orchestra.run.smithi141.stdout:[mds.a] 2023-06-23T00:35:03.932 INFO:teuthology.orchestra.run.smithi141.stdout:[mds.c] 2023-06-23T00:35:03.932 INFO:teuthology.orchestra.run.smithi141.stdout:[mds.b] 2023-06-23T00:35:03.932 INFO:teuthology.orchestra.run.smithi141.stdout:[mds.d] 2023-06-23T00:35:03.933 INFO:teuthology.orchestra.run.smithi141.stdout:[mds] 2023-06-23T00:35:03.933 INFO:teuthology.orchestra.run.smithi141.stdout: debug mds = 20 2023-06-23T00:35:03.933 INFO:teuthology.orchestra.run.smithi141.stdout: debug ms = 1 2023-06-23T00:35:03.933 INFO:teuthology.orchestra.run.smithi141.stdout: mds debug frag = True 2023-06-23T00:35:03.933 INFO:teuthology.orchestra.run.smithi141.stdout: mds debug scatterstat = True 2023-06-23T00:35:03.933 INFO:teuthology.orchestra.run.smithi141.stdout: mds op complaint time = 180 2023-06-23T00:35:03.934 INFO:teuthology.orchestra.run.smithi141.stdout: mds verify scatter = True 2023-06-23T00:35:03.934 INFO:teuthology.orchestra.run.smithi141.stdout: osd op complaint time = 180 2023-06-23T00:35:03.934 INFO:teuthology.orchestra.run.smithi141.stdout: rados mon op timeout = 900 2023-06-23T00:35:03.934 INFO:teuthology.orchestra.run.smithi141.stdout: rados osd op timeout = 900 2023-06-23T00:35:03.934 INFO:tasks.ceph:Creating admin key on mon.a... 2023-06-23T00:35:03.935 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-06-23T00:35:04.001 INFO:tasks.ceph:Copying monmap to all nodes... 2023-06-23T00:35:04.001 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:04.001 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-06-23T00:35:04.016 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:04.017 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-06-23T00:35:04.071 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:35:04.071 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:04.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-23T00:35:04.072 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T00:35:04.147 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:04.147 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:04.201 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:35:04.202 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:04.202 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-23T00:35:04.203 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T00:35:04.237 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:04.238 DEBUG:teuthology.orchestra.run.smithi141:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:04.293 INFO:tasks.ceph:Setting up mon nodes... 2023-06-23T00:35:04.294 INFO:tasks.ceph:Setting up mgr nodes... 2023-06-23T00:35:04.294 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-06-23T00:35:04.344 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-06-23T00:35:04.346 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-06-23T00:35:04.405 INFO:teuthology.orchestra.run.smithi141.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-06-23T00:35:04.407 INFO:tasks.ceph:Setting up mds nodes... 2023-06-23T00:35:04.408 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-06-23T00:35:04.458 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-06-23T00:35:04.460 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-06-23T00:35:04.486 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-06-23T00:35:04.576 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-06-23T00:35:04.578 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-06-23T00:35:04.603 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-06-23T00:35:04.651 INFO:teuthology.orchestra.run.smithi141.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-06-23T00:35:04.653 DEBUG:teuthology.orchestra.run.smithi141:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-06-23T00:35:04.678 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-06-23T00:35:04.765 INFO:teuthology.orchestra.run.smithi141.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-06-23T00:35:04.767 DEBUG:teuthology.orchestra.run.smithi141:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-06-23T00:35:04.792 INFO:tasks.ceph_client:Setting up client nodes... 2023-06-23T00:35:04.792 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-23T00:35:04.832 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-06-23T00:35:04.845 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-06-23T00:35:04.924 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-06-23T00:35:04.938 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-06-23T00:35:04.977 INFO:teuthology.orchestra.run.smithi141.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-06-23T00:35:04.989 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-06-23T00:35:05.067 INFO:teuthology.orchestra.run.smithi141.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-06-23T00:35:05.079 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-06-23T00:35:05.080 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi055.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi141.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-23T00:35:05.080 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-06-23T00:35:05.105 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T00:35:05.106 INFO:tasks.ceph:role: osd.0 2023-06-23T00:35:05.106 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:35:05.106 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-23T00:35:05.199 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T00:35:05.199 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T00:35:05.200 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T00:35:05.200 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2023-06-23T00:35:05.200 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T00:35:05.200 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2023-06-23T00:35:05.201 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T00:35:05.201 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T00:35:05.201 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T00:35:05.201 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T00:35:06.476 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2023-06-23T00:35:06.478 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2023-06-23T00:35:06.478 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-06-23T00:35:06.527 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-06-23T00:35:06.560 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-06-23T00:35:06.626 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T00:35:06.627 INFO:tasks.ceph:role: osd.1 2023-06-23T00:35:06.627 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:35:06.627 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-23T00:35:06.722 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T00:35:06.722 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T00:35:06.722 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T00:35:06.722 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2023-06-23T00:35:06.723 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T00:35:06.723 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2023-06-23T00:35:06.723 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T00:35:06.723 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T00:35:06.723 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T00:35:06.723 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T00:35:07.997 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2023-06-23T00:35:07.999 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2023-06-23T00:35:08.000 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-06-23T00:35:08.046 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-06-23T00:35:08.078 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-06-23T00:35:08.145 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T00:35:08.146 INFO:tasks.ceph:role: osd.2 2023-06-23T00:35:08.146 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:35:08.146 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-23T00:35:08.240 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T00:35:08.240 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T00:35:08.241 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T00:35:08.241 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2023-06-23T00:35:08.241 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T00:35:08.241 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2023-06-23T00:35:08.241 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T00:35:08.241 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T00:35:08.241 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T00:35:08.241 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T00:35:09.516 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2023-06-23T00:35:09.518 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2023-06-23T00:35:09.518 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-06-23T00:35:09.566 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-06-23T00:35:09.598 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-06-23T00:35:09.665 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T00:35:09.666 INFO:tasks.ceph:role: osd.3 2023-06-23T00:35:09.666 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:35:09.666 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-23T00:35:09.763 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T00:35:09.763 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T00:35:09.764 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T00:35:09.764 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2023-06-23T00:35:09.764 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T00:35:09.765 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2023-06-23T00:35:09.765 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T00:35:09.765 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T00:35:09.766 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T00:35:09.766 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T00:35:11.038 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2023-06-23T00:35:11.040 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2023-06-23T00:35:11.041 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-06-23T00:35:11.087 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-06-23T00:35:11.119 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:11.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:11.225+0000 7f42e45b6540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-06-23T00:35:11.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:11.225+0000 7f42e45b6540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-06-23T00:35:11.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:11.225+0000 7f42e45b6540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T00:35:11.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:11.225+0000 7f42e45b6540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T00:35:11.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:11.225+0000 7f42e45b6540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T00:35:11.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:11.226+0000 7f42e45b6540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-06-23T00:35:11.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:11.751+0000 7f42e45b6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:12.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:12.514+0000 7f42e45b6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:13.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:13.278+0000 7f42e45b6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:14.082 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-06-23T00:35:14.113 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:14.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:14.200+0000 7fd40ea1d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-06-23T00:35:14.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:14.200+0000 7fd40ea1d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-06-23T00:35:14.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:14.200+0000 7fd40ea1d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T00:35:14.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:14.200+0000 7fd40ea1d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T00:35:14.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:14.200+0000 7fd40ea1d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T00:35:14.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:14.201+0000 7fd40ea1d540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-06-23T00:35:14.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:14.726+0000 7fd40ea1d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:15.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:15.485+0000 7fd40ea1d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:16.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:16.248+0000 7fd40ea1d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:17.063 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-06-23T00:35:17.094 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:17.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:17.178+0000 7f0cf9e4e540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-06-23T00:35:17.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:17.178+0000 7f0cf9e4e540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-06-23T00:35:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:17.178+0000 7f0cf9e4e540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T00:35:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:17.178+0000 7f0cf9e4e540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T00:35:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:17.178+0000 7f0cf9e4e540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T00:35:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:17.179+0000 7f0cf9e4e540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-06-23T00:35:17.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:17.704+0000 7f0cf9e4e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:18.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:18.471+0000 7f0cf9e4e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:19.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:19.232+0000 7f0cf9e4e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:20.056 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-06-23T00:35:20.087 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:20.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:20.177+0000 7f8967c6b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-06-23T00:35:20.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:20.177+0000 7f8967c6b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-06-23T00:35:20.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:20.177+0000 7f8967c6b540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T00:35:20.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:20.177+0000 7f8967c6b540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T00:35:20.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:20.177+0000 7f8967c6b540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T00:35:20.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:20.178+0000 7f8967c6b540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-06-23T00:35:20.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:20.701+0000 7f8967c6b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:21.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:21.466+0000 7f8967c6b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:22.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:22.228+0000 7f8967c6b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:23.042 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-06-23T00:35:23.072 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-06-23T00:35:23.100 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T00:35:23.100 INFO:tasks.ceph:role: osd.4 2023-06-23T00:35:23.100 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:35:23.101 DEBUG:teuthology.orchestra.run.smithi141:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-23T00:35:23.173 INFO:teuthology.orchestra.run.smithi141.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T00:35:23.174 INFO:teuthology.orchestra.run.smithi141.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T00:35:23.174 INFO:teuthology.orchestra.run.smithi141.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T00:35:23.174 INFO:teuthology.orchestra.run.smithi141.stdout: = reflink=1 2023-06-23T00:35:23.174 INFO:teuthology.orchestra.run.smithi141.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T00:35:23.174 INFO:teuthology.orchestra.run.smithi141.stdout: = sunit=0 swidth=0 blks 2023-06-23T00:35:23.174 INFO:teuthology.orchestra.run.smithi141.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T00:35:23.174 INFO:teuthology.orchestra.run.smithi141.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T00:35:23.174 INFO:teuthology.orchestra.run.smithi141.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T00:35:23.175 INFO:teuthology.orchestra.run.smithi141.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T00:35:23.598 INFO:teuthology.orchestra.run.smithi141.stdout:Discarding blocks...Done. 2023-06-23T00:35:23.600 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi141.front.sepia.ceph.com -o noatime 2023-06-23T00:35:23.600 DEBUG:teuthology.orchestra.run.smithi141:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-06-23T00:35:23.787 DEBUG:teuthology.orchestra.run.smithi141:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-06-23T00:35:23.819 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-06-23T00:35:23.884 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T00:35:23.884 INFO:tasks.ceph:role: osd.5 2023-06-23T00:35:23.885 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:35:23.885 DEBUG:teuthology.orchestra.run.smithi141:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-23T00:35:23.956 INFO:teuthology.orchestra.run.smithi141.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T00:35:23.956 INFO:teuthology.orchestra.run.smithi141.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T00:35:23.956 INFO:teuthology.orchestra.run.smithi141.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T00:35:23.956 INFO:teuthology.orchestra.run.smithi141.stdout: = reflink=1 2023-06-23T00:35:23.957 INFO:teuthology.orchestra.run.smithi141.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T00:35:23.957 INFO:teuthology.orchestra.run.smithi141.stdout: = sunit=0 swidth=0 blks 2023-06-23T00:35:23.957 INFO:teuthology.orchestra.run.smithi141.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T00:35:23.957 INFO:teuthology.orchestra.run.smithi141.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T00:35:23.957 INFO:teuthology.orchestra.run.smithi141.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T00:35:23.957 INFO:teuthology.orchestra.run.smithi141.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T00:35:24.381 INFO:teuthology.orchestra.run.smithi141.stdout:Discarding blocks...Done. 2023-06-23T00:35:24.383 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi141.front.sepia.ceph.com -o noatime 2023-06-23T00:35:24.383 DEBUG:teuthology.orchestra.run.smithi141:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-06-23T00:35:24.489 DEBUG:teuthology.orchestra.run.smithi141:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-06-23T00:35:24.516 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-06-23T00:35:24.581 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T00:35:24.581 INFO:tasks.ceph:role: osd.6 2023-06-23T00:35:24.581 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:35:24.582 DEBUG:teuthology.orchestra.run.smithi141:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-23T00:35:24.652 INFO:teuthology.orchestra.run.smithi141.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T00:35:24.652 INFO:teuthology.orchestra.run.smithi141.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T00:35:24.653 INFO:teuthology.orchestra.run.smithi141.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T00:35:24.653 INFO:teuthology.orchestra.run.smithi141.stdout: = reflink=1 2023-06-23T00:35:24.653 INFO:teuthology.orchestra.run.smithi141.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T00:35:24.653 INFO:teuthology.orchestra.run.smithi141.stdout: = sunit=0 swidth=0 blks 2023-06-23T00:35:24.653 INFO:teuthology.orchestra.run.smithi141.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T00:35:24.653 INFO:teuthology.orchestra.run.smithi141.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T00:35:24.653 INFO:teuthology.orchestra.run.smithi141.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T00:35:24.654 INFO:teuthology.orchestra.run.smithi141.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T00:35:25.099 INFO:teuthology.orchestra.run.smithi141.stdout:Discarding blocks...Done. 2023-06-23T00:35:25.101 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi141.front.sepia.ceph.com -o noatime 2023-06-23T00:35:25.101 DEBUG:teuthology.orchestra.run.smithi141:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-06-23T00:35:25.255 DEBUG:teuthology.orchestra.run.smithi141:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-06-23T00:35:25.285 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-06-23T00:35:25.349 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T00:35:25.349 INFO:tasks.ceph:role: osd.7 2023-06-23T00:35:25.350 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:35:25.350 DEBUG:teuthology.orchestra.run.smithi141:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-23T00:35:25.421 INFO:teuthology.orchestra.run.smithi141.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T00:35:25.422 INFO:teuthology.orchestra.run.smithi141.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T00:35:25.422 INFO:teuthology.orchestra.run.smithi141.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T00:35:25.422 INFO:teuthology.orchestra.run.smithi141.stdout: = reflink=1 2023-06-23T00:35:25.422 INFO:teuthology.orchestra.run.smithi141.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T00:35:25.422 INFO:teuthology.orchestra.run.smithi141.stdout: = sunit=0 swidth=0 blks 2023-06-23T00:35:25.423 INFO:teuthology.orchestra.run.smithi141.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T00:35:25.423 INFO:teuthology.orchestra.run.smithi141.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T00:35:25.423 INFO:teuthology.orchestra.run.smithi141.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T00:35:25.423 INFO:teuthology.orchestra.run.smithi141.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T00:35:25.867 INFO:teuthology.orchestra.run.smithi141.stdout:Discarding blocks...Done. 2023-06-23T00:35:25.869 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi141.front.sepia.ceph.com -o noatime 2023-06-23T00:35:25.869 DEBUG:teuthology.orchestra.run.smithi141:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-06-23T00:35:26.045 DEBUG:teuthology.orchestra.run.smithi141:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-06-23T00:35:26.074 DEBUG:teuthology.orchestra.run.smithi141:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:26.165 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:26.162+0000 7f58bf46d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-06-23T00:35:26.165 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:26.162+0000 7f58bf46d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-06-23T00:35:26.166 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:26.162+0000 7f58bf46d540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T00:35:26.166 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:26.162+0000 7f58bf46d540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T00:35:26.166 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:26.162+0000 7f58bf46d540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T00:35:26.166 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:26.163+0000 7f58bf46d540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-06-23T00:35:26.687 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:26.684+0000 7f58bf46d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:27.452 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:27.449+0000 7f58bf46d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:28.214 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:28.211+0000 7f58bf46d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:29.015 DEBUG:teuthology.orchestra.run.smithi141:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-06-23T00:35:29.042 DEBUG:teuthology.orchestra.run.smithi141:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:29.126 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:29.122+0000 7f2abb045540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-06-23T00:35:29.126 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:29.123+0000 7f2abb045540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-06-23T00:35:29.126 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:29.123+0000 7f2abb045540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T00:35:29.126 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:29.123+0000 7f2abb045540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T00:35:29.127 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:29.123+0000 7f2abb045540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T00:35:29.127 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:29.123+0000 7f2abb045540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-06-23T00:35:29.656 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:29.652+0000 7f2abb045540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:30.417 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:30.414+0000 7f2abb045540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:31.179 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:31.176+0000 7f2abb045540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:31.834 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T00:35:31.837 DEBUG:teuthology.orchestra.run.smithi141:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T00:35:31.980 DEBUG:teuthology.orchestra.run.smithi141:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-06-23T00:35:32.048 DEBUG:teuthology.orchestra.run.smithi141:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:32.131 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:32.128+0000 7f568f20d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-06-23T00:35:32.132 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:32.128+0000 7f568f20d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-06-23T00:35:32.132 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:32.128+0000 7f568f20d540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T00:35:32.132 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:32.129+0000 7f568f20d540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T00:35:32.132 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:32.129+0000 7f568f20d540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T00:35:32.133 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:32.129+0000 7f568f20d540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-06-23T00:35:32.653 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:32.651+0000 7f568f20d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:33.415 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:33.412+0000 7f568f20d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:34.177 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:34.174+0000 7f568f20d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:34.981 DEBUG:teuthology.orchestra.run.smithi141:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-06-23T00:35:35.051 DEBUG:teuthology.orchestra.run.smithi141:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:35.139 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:35.135+0000 7f0930af5540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-06-23T00:35:35.139 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:35.135+0000 7f0930af5540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-06-23T00:35:35.140 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:35.136+0000 7f0930af5540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T00:35:35.140 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:35.136+0000 7f0930af5540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T00:35:35.140 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:35.136+0000 7f0930af5540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T00:35:35.140 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:35.136+0000 7f0930af5540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-06-23T00:35:35.664 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:35.661+0000 7f0930af5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:36.427 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:36.424+0000 7f0930af5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:37.189 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:37.186+0000 7f0930af5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T00:35:37.983 DEBUG:teuthology.orchestra.run.smithi141:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-06-23T00:35:38.010 INFO:tasks.ceph:Reading keys from all nodes... 2023-06-23T00:35:38.010 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:38.011 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-06-23T00:35:38.041 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:38.042 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-06-23T00:35:38.110 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:38.110 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-06-23T00:35:38.176 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:38.177 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-06-23T00:35:38.243 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:38.243 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-06-23T00:35:38.308 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:38.309 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-06-23T00:35:38.374 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:38.374 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-06-23T00:35:38.439 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:38.439 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-06-23T00:35:38.466 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:38.466 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-06-23T00:35:38.532 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:38.532 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-06-23T00:35:38.597 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:38.597 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-06-23T00:35:38.662 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:38.663 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-06-23T00:35:38.728 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:38.728 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-06-23T00:35:38.792 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:38.792 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-06-23T00:35:38.856 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:38.857 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-06-23T00:35:38.873 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:38.874 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-06-23T00:35:38.929 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:38.930 DEBUG:teuthology.orchestra.run.smithi141:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-06-23T00:35:38.945 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:38.946 DEBUG:teuthology.orchestra.run.smithi141:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-06-23T00:35:39.000 INFO:tasks.ceph:Adding keys to all mons... 2023-06-23T00:35:39.001 DEBUG:teuthology.orchestra.run.smithi055:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-23T00:35:39.004 DEBUG:teuthology.orchestra.run.smithi141:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-23T00:35:39.043 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr.y] 2023-06-23T00:35:39.044 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC46JRkuyVoFBAAxIluoSpoRN7AdY3K0ORcwg== 2023-06-23T00:35:39.044 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2023-06-23T00:35:39.044 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC46JRk11Q5GxAAv5ogsoSVvSfL5b0pTD+t2w== 2023-06-23T00:35:39.044 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2023-06-23T00:35:39.044 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC46JRk4FY2IhAAFxHnrYEMnBatFwelm0WjtA== 2023-06-23T00:35:39.045 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.0] 2023-06-23T00:35:39.045 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC/6JRkuyp0DRAA5nevVIuOuaLap6C+nClJuQ== 2023-06-23T00:35:39.045 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.1] 2023-06-23T00:35:39.045 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDC6JRkC7n9CxAAqlAaPX6tQnqwvR8WBFutAg== 2023-06-23T00:35:39.045 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.2] 2023-06-23T00:35:39.046 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDF6JRkjO2sChAAJPnS/lpNFo0aVvJyIwXWWQ== 2023-06-23T00:35:39.046 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.3] 2023-06-23T00:35:39.046 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDI6JRkQ8GbChAAE+karyyzQXujzlYsuWr78Q== 2023-06-23T00:35:39.047 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr.x] 2023-06-23T00:35:39.047 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC46JRkWkILGBAAMg49XjU08924NchPq+3wBA== 2023-06-23T00:35:39.047 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2023-06-23T00:35:39.047 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC46JRkHLS6JhAAYIpjzkNpC6zSdR16W7KJ0A== 2023-06-23T00:35:39.047 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2023-06-23T00:35:39.047 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC46JRkVCOBLRAASq00RywQk3OGNAgLVfTUhA== 2023-06-23T00:35:39.048 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.4] 2023-06-23T00:35:39.048 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDO6JRk1Um7CRAA478zKiuzfLJXh36O3ylh7Q== 2023-06-23T00:35:39.048 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.5] 2023-06-23T00:35:39.048 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDR6JRkZD9iBxAAlg5ohdXdwCflWOIUb/nGuw== 2023-06-23T00:35:39.048 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.6] 2023-06-23T00:35:39.049 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDU6JRkHyi5BxAA5KJ8S2oxotFBi1HLxrWbwg== 2023-06-23T00:35:39.049 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.7] 2023-06-23T00:35:39.049 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDX6JRkqGImCBAAlMAP1xaCrRplH7PS8YHZ7g== 2023-06-23T00:35:39.049 INFO:teuthology.orchestra.run.smithi055.stdout:[client.0] 2023-06-23T00:35:39.049 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC46JRkH76FMRAArP2PwU7/H/vyiF2nYMANlA== 2023-06-23T00:35:39.049 INFO:teuthology.orchestra.run.smithi055.stdout:[client.2] 2023-06-23T00:35:39.049 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC46JRkOAP/NhAA2EsJaNmwlyCtFBG/SekiZA== 2023-06-23T00:35:39.050 INFO:teuthology.orchestra.run.smithi055.stdout:[client.1] 2023-06-23T00:35:39.050 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC46JRkPgwiOhAAftvn3uUCHub6cDjhDcpVBA== 2023-06-23T00:35:39.050 INFO:teuthology.orchestra.run.smithi055.stdout:[client.3] 2023-06-23T00:35:39.050 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC56JRkXUPlAxAABRLZddfP+Cu8OhsoSxAF1A== 2023-06-23T00:35:39.063 INFO:teuthology.orchestra.run.smithi141.stdout:[mgr.y] 2023-06-23T00:35:39.064 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC46JRkuyVoFBAAxIluoSpoRN7AdY3K0ORcwg== 2023-06-23T00:35:39.064 INFO:teuthology.orchestra.run.smithi141.stdout:[mds.a] 2023-06-23T00:35:39.064 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC46JRk11Q5GxAAv5ogsoSVvSfL5b0pTD+t2w== 2023-06-23T00:35:39.065 INFO:teuthology.orchestra.run.smithi141.stdout:[mds.c] 2023-06-23T00:35:39.065 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC46JRk4FY2IhAAFxHnrYEMnBatFwelm0WjtA== 2023-06-23T00:35:39.065 INFO:teuthology.orchestra.run.smithi141.stdout:[osd.0] 2023-06-23T00:35:39.065 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC/6JRkuyp0DRAA5nevVIuOuaLap6C+nClJuQ== 2023-06-23T00:35:39.066 INFO:teuthology.orchestra.run.smithi141.stdout:[osd.1] 2023-06-23T00:35:39.066 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQDC6JRkC7n9CxAAqlAaPX6tQnqwvR8WBFutAg== 2023-06-23T00:35:39.066 INFO:teuthology.orchestra.run.smithi141.stdout:[osd.2] 2023-06-23T00:35:39.066 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQDF6JRkjO2sChAAJPnS/lpNFo0aVvJyIwXWWQ== 2023-06-23T00:35:39.066 INFO:teuthology.orchestra.run.smithi141.stdout:[osd.3] 2023-06-23T00:35:39.067 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQDI6JRkQ8GbChAAE+karyyzQXujzlYsuWr78Q== 2023-06-23T00:35:39.067 INFO:teuthology.orchestra.run.smithi141.stdout:[mgr.x] 2023-06-23T00:35:39.067 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC46JRkWkILGBAAMg49XjU08924NchPq+3wBA== 2023-06-23T00:35:39.067 INFO:teuthology.orchestra.run.smithi141.stdout:[mds.b] 2023-06-23T00:35:39.067 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC46JRkHLS6JhAAYIpjzkNpC6zSdR16W7KJ0A== 2023-06-23T00:35:39.067 INFO:teuthology.orchestra.run.smithi141.stdout:[mds.d] 2023-06-23T00:35:39.068 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC46JRkVCOBLRAASq00RywQk3OGNAgLVfTUhA== 2023-06-23T00:35:39.068 INFO:teuthology.orchestra.run.smithi141.stdout:[osd.4] 2023-06-23T00:35:39.068 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQDO6JRk1Um7CRAA478zKiuzfLJXh36O3ylh7Q== 2023-06-23T00:35:39.068 INFO:teuthology.orchestra.run.smithi141.stdout:[osd.5] 2023-06-23T00:35:39.068 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQDR6JRkZD9iBxAAlg5ohdXdwCflWOIUb/nGuw== 2023-06-23T00:35:39.068 INFO:teuthology.orchestra.run.smithi141.stdout:[osd.6] 2023-06-23T00:35:39.069 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQDU6JRkHyi5BxAA5KJ8S2oxotFBi1HLxrWbwg== 2023-06-23T00:35:39.069 INFO:teuthology.orchestra.run.smithi141.stdout:[osd.7] 2023-06-23T00:35:39.069 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQDX6JRkqGImCBAAlMAP1xaCrRplH7PS8YHZ7g== 2023-06-23T00:35:39.069 INFO:teuthology.orchestra.run.smithi141.stdout:[client.0] 2023-06-23T00:35:39.069 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC46JRkH76FMRAArP2PwU7/H/vyiF2nYMANlA== 2023-06-23T00:35:39.069 INFO:teuthology.orchestra.run.smithi141.stdout:[client.2] 2023-06-23T00:35:39.070 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC46JRkOAP/NhAA2EsJaNmwlyCtFBG/SekiZA== 2023-06-23T00:35:39.070 INFO:teuthology.orchestra.run.smithi141.stdout:[client.1] 2023-06-23T00:35:39.070 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC46JRkPgwiOhAAftvn3uUCHub6cDjhDcpVBA== 2023-06-23T00:35:39.070 INFO:teuthology.orchestra.run.smithi141.stdout:[client.3] 2023-06-23T00:35:39.070 INFO:teuthology.orchestra.run.smithi141.stdout: key = AQC56JRkXUPlAxAABRLZddfP+Cu8OhsoSxAF1A== 2023-06-23T00:35:39.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T00:35:39.094 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T00:35:39.159 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T00:35:39.162 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T00:35:39.213 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T00:35:39.215 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T00:35:39.265 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.267 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.314 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.316 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.365 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.367 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.415 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.417 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.464 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T00:35:39.467 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T00:35:39.515 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T00:35:39.517 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T00:35:39.568 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T00:35:39.570 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T00:35:39.620 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.622 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.669 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.671 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.718 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.720 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.767 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.769 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T00:35:39.816 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T00:35:39.820 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T00:35:39.870 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T00:35:39.873 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T00:35:39.923 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T00:35:39.926 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T00:35:39.976 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T00:35:39.979 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T00:35:40.029 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-06-23T00:35:40.029 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-06-23T00:35:40.054 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T00:35:40.162 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-06-23T00:35:40.187 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-06-23T00:35:40.211 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T00:35:40.303 DEBUG:teuthology.orchestra.run.smithi141:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-06-23T00:35:40.327 DEBUG:teuthology.orchestra.run.smithi141:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-06-23T00:35:40.391 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T00:35:40.482 DEBUG:teuthology.orchestra.run.smithi141:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-06-23T00:35:40.507 DEBUG:teuthology.orchestra.run.smithi055:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:40.509 DEBUG:teuthology.orchestra.run.smithi141:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-23T00:35:40.561 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-06-23T00:35:40.561 INFO:tasks.ceph.mon.a:Restarting daemon 2023-06-23T00:35:40.562 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-06-23T00:35:40.564 INFO:tasks.ceph.mon.a:Started 2023-06-23T00:35:40.565 INFO:tasks.ceph.mon.b:Restarting daemon 2023-06-23T00:35:40.565 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-06-23T00:35:40.603 INFO:tasks.ceph.mon.b:Started 2023-06-23T00:35:40.603 INFO:tasks.ceph.mon.c:Restarting daemon 2023-06-23T00:35:40.604 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-06-23T00:35:40.606 INFO:tasks.ceph.mon.c:Started 2023-06-23T00:35:40.607 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-06-23T00:35:40.607 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-06-23T00:35:40.608 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-06-23T00:35:40.610 INFO:tasks.ceph.mgr.y:Started 2023-06-23T00:35:40.611 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-06-23T00:35:40.611 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-06-23T00:35:40.613 INFO:tasks.ceph.mgr.x:Started 2023-06-23T00:35:40.614 INFO:tasks.ceph:Setting crush tunables to default 2023-06-23T00:35:40.614 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd crush tunables default 2023-06-23T00:35:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.731+0000 7f8573a57700 1 Processor -- start 2023-06-23T00:35:40.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.731+0000 7f8573a57700 1 -- start start 2023-06-23T00:35:40.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8573a57700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f856c157480 0x7f856c15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:40.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8573a57700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 0x7f856c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:40.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8573a57700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f856c0b29b0 con 0x7f856c1570b0 2023-06-23T00:35:40.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8573a57700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f856c0d03a0 con 0x7f856c157480 2023-06-23T00:35:40.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8573a57700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856c0cef00 con 0x7f856c0b2010 2023-06-23T00:35:40.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f85717f3700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f856c1570b0 0x7f856c15d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55422/0 (socket says 172.21.15.55:55422) 2023-06-23T00:35:40.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f85717f3700 1 -- 172.21.15.55:0/2866760906 learned_addr learned my addr 172.21.15.55:0/2866760906 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:40.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8570ff2700 1 --2- 172.21.15.55:0/2866760906 >> v2:172.21.15.55:3300/0 conn(0x7f856c157480 0x7f856c15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:40.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8571ff4700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 0x7f856c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:40.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8570ff2700 1 -- 172.21.15.55:0/2866760906 >> v2:172.21.15.55:3300/0 conn(0x7f856c157480 msgr2=0x7f856c15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-23T00:35:40.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8570ff2700 1 -- 172.21.15.55:0/2866760906 >> v2:172.21.15.55:3300/0 conn(0x7f856c157480 msgr2=0x7f856c15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T00:35:40.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8570ff2700 1 --2- 172.21.15.55:0/2866760906 >> v2:172.21.15.55:3300/0 conn(0x7f856c157480 0x7f856c15a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T00:35:40.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8570ff2700 1 --2- 172.21.15.55:0/2866760906 >> v2:172.21.15.55:3300/0 conn(0x7f856c157480 0x7f856c15a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T00:35:40.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8571ff4700 1 -- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 msgr2=0x7f856c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-23T00:35:40.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8571ff4700 1 -- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 msgr2=0x7f856c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T00:35:40.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8571ff4700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 0x7f856c0b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T00:35:40.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.732+0000 7f8571ff4700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 0x7f856c0b2470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T00:35:40.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.920+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199204543 0 0) 0x7f8568001110 con 0x7f856c1570b0 2023-06-23T00:35:40.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.920+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8558003570 con 0x7f856c1570b0 2023-06-23T00:35:40.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2155278738 0 0) 0x7f8558003570 con 0x7f856c1570b0 2023-06-23T00:35:40.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 msgr2=0x7f856c0b2470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T00:35:40.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 0x7f856c0b2470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:40.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 >> v2:172.21.15.55:3300/0 conn(0x7f856c157480 msgr2=0x7f856c15a8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T00:35:40.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 --2- 172.21.15.55:0/2866760906 >> v2:172.21.15.55:3300/0 conn(0x7f856c157480 0x7f856c15a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:40.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c0cec00 con 0x7f856c1570b0 2023-06-23T00:35:40.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2542258539 0 0) 0x7f8568001600 con 0x7f856c1570b0 2023-06-23T00:35:40.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 >> v1:172.21.15.55:6789/0 conn(0x7f856c1570b0 legacy=0x7f856c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:40.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8558003a60 0x7f8558003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:40.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 0x7f8558004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:40.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8558004d50 0x7f85580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:40.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c0d03a0 con 0x7f8558003a60 2023-06-23T00:35:40.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c0cef00 con 0x7f8558004d50 2023-06-23T00:35:40.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855800bbc0 con 0x7f856c0b2010 2023-06-23T00:35:40.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f85717f3700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8558003a60 0x7f8558003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:40.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8570ff2700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 0x7f8558004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:40.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8571ff4700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8558004d50 0x7f85580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:40.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f85717f3700 1 -- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 msgr2=0x7f8558004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:40.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f85717f3700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 0x7f8558004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:40.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f85717f3700 1 -- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8558004d50 msgr2=0x7f85580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:40.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f85717f3700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8558004d50 0x7f85580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:40.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f85717f3700 1 -- 172.21.15.55:0/2866760906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8558003570 con 0x7f8558003a60 2023-06-23T00:35:40.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8570ff2700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c0b2010 0x7f8558004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T00:35:40.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f85717f3700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8558003a60 0x7f8558003e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f856800c5f0 tx=0x7f856c0b6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f98d884d8fc0e1a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:40.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.921+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800cdf0 con 0x7f8558003a60 2023-06-23T00:35:40.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.922+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8568005920 con 0x7f8558003a60 2023-06-23T00:35:40.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.922+0000 7f8563fff700 1 -- 172.21.15.55:0/2866760906 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568005ce0 con 0x7f8558003a60 2023-06-23T00:35:40.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.922+0000 7f8573a57700 1 -- 172.21.15.55:0/2866760906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8558003a60 msgr2=0x7f8558003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:40.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.922+0000 7f8573a57700 1 --2- 172.21.15.55:0/2866760906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8558003a60 0x7f8558003e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f856800c5f0 tx=0x7f856c0b6040 comp rx=0 tx=0).stop 2023-06-23T00:35:40.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.922+0000 7f8571ff4700 1 -- 172.21.15.55:0/2866760906 reap_dead start 2023-06-23T00:35:40.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.922+0000 7f8573a57700 1 -- 172.21.15.55:0/2866760906 shutdown_connections 2023-06-23T00:35:40.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.922+0000 7f8573a57700 1 -- 172.21.15.55:0/2866760906 >> 172.21.15.55:0/2866760906 conn(0x7f856c070290 msgr2=0x7f856c070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:40.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.922+0000 7f8573a57700 1 -- 172.21.15.55:0/2866760906 shutdown_connections 2023-06-23T00:35:40.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.922+0000 7f8573a57700 1 -- 172.21.15.55:0/2866760906 wait complete. 2023-06-23T00:35:40.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.924+0000 7f8573a57700 1 Processor -- start 2023-06-23T00:35:40.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.924+0000 7f8573a57700 1 -- start start 2023-06-23T00:35:40.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.924+0000 7f8573a57700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8558003a60 0x7f856c14ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:40.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.924+0000 7f8573a57700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f856c139720 0x7f856c16ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:40.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.924+0000 7f8573a57700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c139b90 0x7f856c16d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:40.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.924+0000 7f8573a57700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c14e5f0 con 0x7f856c139720 2023-06-23T00:35:40.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.924+0000 7f8573a57700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c14efc0 con 0x7f8558003a60 2023-06-23T00:35:40.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.924+0000 7f8573a57700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856c14b560 con 0x7f856c139b90 2023-06-23T00:35:40.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f8570ff2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f856c139720 0x7f856c16ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f85717f3700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8558003a60 0x7f856c14ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f8570ff2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f856c139720 0x7f856c16ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50714/0 (socket says 172.21.15.55:50714) 2023-06-23T00:35:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f85717f3700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8558003a60 0x7f856c14ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.55:42058/0 (socket says 172.21.15.55:42058) 2023-06-23T00:35:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f8570ff2700 1 -- 172.21.15.55:0/1599914204 learned_addr learned my addr 172.21.15.55:0/1599914204 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f8571ff4700 1 --2- 172.21.15.55:0/1599914204 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c139b90 0x7f856c16d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f8570ff2700 1 -- 172.21.15.55:0/1599914204 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c139b90 msgr2=0x7f856c16d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f8570ff2700 1 --2- 172.21.15.55:0/1599914204 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c139b90 0x7f856c16d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f8570ff2700 1 -- 172.21.15.55:0/1599914204 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8558003a60 msgr2=0x7f856c14ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f8570ff2700 1 --2- 172.21.15.55:0/1599914204 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8558003a60 0x7f856c14ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f8570ff2700 1 -- 172.21.15.55:0/1599914204 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c16da30 con 0x7f856c139720 2023-06-23T00:35:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.925+0000 7f8570ff2700 1 --2- 172.21.15.55:0/1599914204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f856c139720 0x7f856c16ce90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f855c00f910 tx=0x7f855c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.926+0000 7f85617fa700 1 -- 172.21.15.55:0/1599914204 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c014670 con 0x7f856c139720 2023-06-23T00:35:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.926+0000 7f85617fa700 1 -- 172.21.15.55:0/1599914204 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855c0147d0 con 0x7f856c139720 2023-06-23T00:35:40.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.926+0000 7f8573a57700 1 -- 172.21.15.55:0/1599914204 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f856c14dfa0 con 0x7f856c139720 2023-06-23T00:35:40.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.926+0000 7f85617fa700 1 -- 172.21.15.55:0/1599914204 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c014b50 con 0x7f856c139720 2023-06-23T00:35:40.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.926+0000 7f8573a57700 1 -- 172.21.15.55:0/1599914204 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856c0cf6f0 con 0x7f856c139720 2023-06-23T00:35:40.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.926+0000 7f85617fa700 1 -- 172.21.15.55:0/1599914204 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f855c015410 con 0x7f856c139720 2023-06-23T00:35:40.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.926+0000 7f85617fa700 1 -- 172.21.15.55:0/1599914204 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f855c01e750 con 0x7f856c139720 2023-06-23T00:35:40.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.927+0000 7f8573a57700 1 -- 172.21.15.55:0/1599914204 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856c14e8d0 con 0x7f856c139720 2023-06-23T00:35:40.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:40.930+0000 7f85617fa700 1 -- 172.21.15.55:0/1599914204 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f855c03d6e0 con 0x7f856c139720 2023-06-23T00:35:41.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.008+0000 7f8573a57700 1 -- 172.21.15.55:0/1599914204 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f856c04c1b0 con 0x7f856c139720 2023-06-23T00:35:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.014+0000 7f85617fa700 1 -- 172.21.15.55:0/1599914204 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f855c02c020 con 0x7f856c139720 2023-06-23T00:35:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.015+0000 7f8573a57700 1 -- 172.21.15.55:0/1599914204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f856c139720 msgr2=0x7f856c16ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.015+0000 7f8573a57700 1 --2- 172.21.15.55:0/1599914204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f856c139720 0x7f856c16ce90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f855c00f910 tx=0x7f855c008040 comp rx=0 tx=0).stop 2023-06-23T00:35:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.015+0000 7f8573a57700 1 -- 172.21.15.55:0/1599914204 shutdown_connections 2023-06-23T00:35:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.015+0000 7f8573a57700 1 --2- 172.21.15.55:0/1599914204 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f856c139b90 0x7f856c16d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.015+0000 7f8573a57700 1 --2- 172.21.15.55:0/1599914204 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8558003a60 0x7f856c14ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.015+0000 7f8573a57700 1 --2- 172.21.15.55:0/1599914204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f856c139720 0x7f856c16ce90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.015+0000 7f8573a57700 1 -- 172.21.15.55:0/1599914204 >> 172.21.15.55:0/1599914204 conn(0x7f856c070290 msgr2=0x7f856c154ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.015+0000 7f8573a57700 1 -- 172.21.15.55:0/1599914204 shutdown_connections 2023-06-23T00:35:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:41.015+0000 7f8573a57700 1 -- 172.21.15.55:0/1599914204 wait complete. 2023-06-23T00:35:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:adjusted tunables profile to default 2023-06-23T00:35:41.031 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-06-23T00:35:41.031 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:41.031 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-06-23T00:35:41.057 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:41.057 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-06-23T00:35:41.121 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:41.122 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-06-23T00:35:41.186 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-06-23T00:35:41.187 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-06-23T00:35:41.252 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:41.252 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-06-23T00:35:41.276 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:41.276 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-06-23T00:35:41.340 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:41.341 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-06-23T00:35:41.405 DEBUG:teuthology.orchestra.run.smithi141:> set -ex 2023-06-23T00:35:41.405 DEBUG:teuthology.orchestra.run.smithi141:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-06-23T00:35:41.468 DEBUG:teuthology.orchestra.run.smithi141:> sudo ceph --cluster ceph osd new d4590eaf-cc24-4ff1-8f1b-9637138bd300 0 2023-06-23T00:35:41.624 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.621+0000 7f1cf0582700 1 Processor -- start 2023-06-23T00:35:41.625 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.621+0000 7f1cf0582700 1 -- start start 2023-06-23T00:35:41.628 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.622+0000 7f1cf0582700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ce8153a80 0x7f1ce80d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.628 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.622+0000 7f1cf0582700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153e50 0x7f1ce815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.628 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.622+0000 7f1cf0582700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cedd0 con 0x7f1ce80b2010 2023-06-23T00:35:41.629 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.622+0000 7f1cf0582700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80d14c0 con 0x7f1ce8153a80 2023-06-23T00:35:41.629 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.622+0000 7f1cf0582700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80ceac0 con 0x7f1ce8153e50 2023-06-23T00:35:41.629 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.622+0000 7f1cedb1d700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153e50 0x7f1ce815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.629 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.622+0000 7f1cedb1d700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153e50 0x7f1ce815a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3301/0 says I am v2:172.21.15.141:50292/0 (socket says 172.21.15.141:50292) 2023-06-23T00:35:41.629 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.622+0000 7f1cedb1d700 1 -- 172.21.15.141:0/3347297937 learned_addr learned my addr 172.21.15.141:0/3347297937 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:41.629 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.623+0000 7f1cee31e700 1 --2- 172.21.15.141:0/3347297937 >> v2:172.21.15.55:3300/0 conn(0x7f1ce8153a80 0x7f1ce80d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.630 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.623+0000 7f1cedb1d700 1 -- 172.21.15.141:0/3347297937 >> v2:172.21.15.55:3300/0 conn(0x7f1ce8153a80 msgr2=0x7f1ce80d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:41.630 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.623+0000 7f1cedb1d700 1 --2- 172.21.15.141:0/3347297937 >> v2:172.21.15.55:3300/0 conn(0x7f1ce8153a80 0x7f1ce80d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.630 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.623+0000 7f1cedb1d700 1 -- 172.21.15.141:0/3347297937 >> v1:172.21.15.55:6789/0 conn(0x7f1ce80b2010 legacy=0x7f1ce815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:41.630 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.623+0000 7f1cedb1d700 1 -- 172.21.15.141:0/3347297937 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce80d0c20 con 0x7f1ce8153e50 2023-06-23T00:35:41.630 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.623+0000 7f1cedb1d700 1 --2- 172.21.15.141:0/3347297937 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153e50 0x7f1ce815a7c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40099b0 tx=0x7f1ce4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdb44c7999445fa4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:41.630 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.623+0000 7f1ced31c700 1 -- 172.21.15.141:0/3347297937 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4009ea0 con 0x7f1ce8153e50 2023-06-23T00:35:41.630 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.623+0000 7f1ced31c700 1 -- 172.21.15.141:0/3347297937 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce400b860 con 0x7f1ce8153e50 2023-06-23T00:35:41.630 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.623+0000 7f1ced31c700 1 -- 172.21.15.141:0/3347297937 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400bc00 con 0x7f1ce8153e50 2023-06-23T00:35:41.631 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.624+0000 7f1cf0582700 1 -- 172.21.15.141:0/3347297937 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153e50 msgr2=0x7f1ce815a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:41.631 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.624+0000 7f1cf0582700 1 --2- 172.21.15.141:0/3347297937 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153e50 0x7f1ce815a7c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40099b0 tx=0x7f1ce4009250 comp rx=0 tx=0).stop 2023-06-23T00:35:41.631 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.624+0000 7f1cf0582700 1 -- 172.21.15.141:0/3347297937 shutdown_connections 2023-06-23T00:35:41.631 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.624+0000 7f1cf0582700 1 --2- 172.21.15.141:0/3347297937 >> v2:172.21.15.55:3300/0 conn(0x7f1ce8153a80 0x7f1ce80d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.631 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.624+0000 7f1cf0582700 1 --2- 172.21.15.141:0/3347297937 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153e50 0x7f1ce815a7c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.631 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.624+0000 7f1cf0582700 1 -- 172.21.15.141:0/3347297937 >> 172.21.15.141:0/3347297937 conn(0x7f1ce8070290 msgr2=0x7f1ce8070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:41.631 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.624+0000 7f1cf0582700 1 -- 172.21.15.141:0/3347297937 shutdown_connections 2023-06-23T00:35:41.632 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.624+0000 7f1cf0582700 1 -- 172.21.15.141:0/3347297937 wait complete. 2023-06-23T00:35:41.632 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.625+0000 7f1cf0582700 1 Processor -- start 2023-06-23T00:35:41.632 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.625+0000 7f1cf0582700 1 -- start start 2023-06-23T00:35:41.634 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cf0582700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ce80b2010 0x7f1ce815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.634 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cf0582700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153a80 0x7f1ce814ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.634 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cf0582700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1ce8153e50 0x7f1ce814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.635 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cf0582700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8159de0 con 0x7f1ce80b2010 2023-06-23T00:35:41.635 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cf0582700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce815a0c0 con 0x7f1ce8153e50 2023-06-23T00:35:41.635 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cf0582700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce815a3a0 con 0x7f1ce8153a80 2023-06-23T00:35:41.635 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cedb1d700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153a80 0x7f1ce814ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.635 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cedb1d700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153a80 0x7f1ce814ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3301/0 says I am v2:172.21.15.141:50302/0 (socket says 172.21.15.141:50302) 2023-06-23T00:35:41.635 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cedb1d700 1 -- 172.21.15.141:0/2319217654 learned_addr learned my addr 172.21.15.141:0/2319217654 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:41.635 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1ceeb1f700 1 --2- 172.21.15.141:0/2319217654 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1ce8153e50 0x7f1ce814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.636 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cee31e700 1 --2- 172.21.15.141:0/2319217654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ce80b2010 0x7f1ce815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.636 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cedb1d700 1 -- 172.21.15.141:0/2319217654 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1ce8153e50 msgr2=0x7f1ce814f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:41.636 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cedb1d700 1 --2- 172.21.15.141:0/2319217654 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1ce8153e50 0x7f1ce814f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.636 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cedb1d700 1 -- 172.21.15.141:0/2319217654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ce80b2010 msgr2=0x7f1ce815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:41.637 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cedb1d700 1 --2- 172.21.15.141:0/2319217654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ce80b2010 0x7f1ce815ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.637 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cedb1d700 1 -- 172.21.15.141:0/2319217654 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce40094f0 con 0x7f1ce8153a80 2023-06-23T00:35:41.637 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1ceeb1f700 1 --2- 172.21.15.141:0/2319217654 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1ce8153e50 0x7f1ce814f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T00:35:41.637 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cedb1d700 1 --2- 172.21.15.141:0/2319217654 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153a80 0x7f1ce814ef00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80cec00 tx=0x7f1ce4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:41.637 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cde7fc700 1 -- 172.21.15.141:0/2319217654 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400bbe0 con 0x7f1ce8153a80 2023-06-23T00:35:41.637 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cde7fc700 1 -- 172.21.15.141:0/2319217654 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce4025440 con 0x7f1ce8153a80 2023-06-23T00:35:41.637 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cde7fc700 1 -- 172.21.15.141:0/2319217654 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40257e0 con 0x7f1ce8153a80 2023-06-23T00:35:41.638 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.626+0000 7f1cf0582700 1 -- 172.21.15.141:0/2319217654 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce815a680 con 0x7f1ce8153a80 2023-06-23T00:35:41.638 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.627+0000 7f1cf0582700 1 -- 172.21.15.141:0/2319217654 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce815b3c0 con 0x7f1ce8153a80 2023-06-23T00:35:41.638 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.627+0000 7f1cde7fc700 1 -- 172.21.15.141:0/2319217654 <== mon.2 v2:172.21.15.141:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1ce402e440 con 0x7f1ce8153a80 2023-06-23T00:35:41.638 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.627+0000 7f1cde7fc700 1 -- 172.21.15.141:0/2319217654 <== mon.2 v2:172.21.15.141:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1ce4011770 con 0x7f1ce8153a80 2023-06-23T00:35:41.638 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.627+0000 7f1cf0582700 1 -- 172.21.15.141:0/2319217654 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce815ab20 con 0x7f1ce8153a80 2023-06-23T00:35:41.639 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.632+0000 7f1cde7fc700 1 -- 172.21.15.141:0/2319217654 <== mon.2 v2:172.21.15.141:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f1ce4015020 con 0x7f1ce8153a80 2023-06-23T00:35:41.716 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.713+0000 7f1cf0582700 1 -- 172.21.15.141:0/2319217654 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d4590eaf-cc24-4ff1-8f1b-9637138bd300", "id": 0} v 0) v1 -- 0x7f1ce804c1b0 con 0x7f1ce8153a80 2023-06-23T00:35:41.723 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.720+0000 7f1cde7fc700 1 -- 172.21.15.141:0/2319217654 <== mon.2 v2:172.21.15.141:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4590eaf-cc24-4ff1-8f1b-9637138bd300", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1ce402e8d0 con 0x7f1ce8153a80 2023-06-23T00:35:41.724 INFO:teuthology.orchestra.run.smithi141.stdout:0 2023-06-23T00:35:41.725 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.721+0000 7f1cf0582700 1 -- 172.21.15.141:0/2319217654 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153a80 msgr2=0x7f1ce814ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:41.725 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.721+0000 7f1cf0582700 1 --2- 172.21.15.141:0/2319217654 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153a80 0x7f1ce814ef00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80cec00 tx=0x7f1ce4017000 comp rx=0 tx=0).stop 2023-06-23T00:35:41.725 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.721+0000 7f1cf0582700 1 -- 172.21.15.141:0/2319217654 shutdown_connections 2023-06-23T00:35:41.725 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.721+0000 7f1cf0582700 1 --2- 172.21.15.141:0/2319217654 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1ce8153e50 0x7f1ce814f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.725 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.721+0000 7f1cf0582700 1 --2- 172.21.15.141:0/2319217654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ce80b2010 0x7f1ce815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.725 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.721+0000 7f1cf0582700 1 --2- 172.21.15.141:0/2319217654 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1ce8153a80 0x7f1ce814ef00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.726 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.721+0000 7f1cf0582700 1 -- 172.21.15.141:0/2319217654 >> 172.21.15.141:0/2319217654 conn(0x7f1ce8070290 msgr2=0x7f1ce815ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:41.726 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.722+0000 7f1cf0582700 1 -- 172.21.15.141:0/2319217654 shutdown_connections 2023-06-23T00:35:41.726 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.722+0000 7f1cf0582700 1 -- 172.21.15.141:0/2319217654 wait complete. 2023-06-23T00:35:41.736 DEBUG:teuthology.orchestra.run.smithi141:> sudo ceph --cluster ceph osd new 518b1bc8-fb1c-405f-8346-fda7978829a6 1 2023-06-23T00:35:41.851 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.848+0000 7f493fceb700 1 Processor -- start 2023-06-23T00:35:41.851 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.848+0000 7f493fceb700 1 -- start start 2023-06-23T00:35:41.853 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493fceb700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4938153a80 0x7f49380d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.854 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493fceb700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f4938153e50 0x7f493815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.854 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493fceb700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f49380b2010 0x7f49380b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.854 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493fceb700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49380d14c0 con 0x7f4938153a80 2023-06-23T00:35:41.854 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493fceb700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49380b29d0 con 0x7f49380b2010 2023-06-23T00:35:41.854 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493fceb700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49380cedd0 con 0x7f4938153e50 2023-06-23T00:35:41.854 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493e288700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f49380b2010 0x7f49380b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.854 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493d286700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f4938153e50 0x7f493815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.855 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493d286700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f4938153e50 0x7f493815a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3301/0 says I am v2:172.21.15.141:50318/0 (socket says 172.21.15.141:50318) 2023-06-23T00:35:41.855 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493e288700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f49380b2010 0x7f49380b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.141:45722/0 (socket says 172.21.15.141:45722) 2023-06-23T00:35:41.855 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493d286700 1 -- 172.21.15.141:0/884869526 learned_addr learned my addr 172.21.15.141:0/884869526 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:41.855 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493da87700 1 --2- 172.21.15.141:0/884869526 >> v2:172.21.15.55:3300/0 conn(0x7f4938153a80 0x7f49380d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.855 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493d286700 1 -- 172.21.15.141:0/884869526 >> v2:172.21.15.141:3300/0 conn(0x7f49380b2010 msgr2=0x7f49380b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:41.855 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493d286700 1 --2- 172.21.15.141:0/884869526 >> v2:172.21.15.141:3300/0 conn(0x7f49380b2010 0x7f49380b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.855 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493d286700 1 -- 172.21.15.141:0/884869526 >> v2:172.21.15.55:3300/0 conn(0x7f4938153a80 msgr2=0x7f49380d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:41.856 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493d286700 1 --2- 172.21.15.141:0/884869526 >> v2:172.21.15.55:3300/0 conn(0x7f4938153a80 0x7f49380d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.856 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493d286700 1 -- 172.21.15.141:0/884869526 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49380ceac0 con 0x7f4938153e50 2023-06-23T00:35:41.856 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493e288700 1 --2- 172.21.15.141:0/884869526 >> v2:172.21.15.141:3300/0 conn(0x7f49380b2010 0x7f49380b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:41.856 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.849+0000 7f493d286700 1 --2- 172.21.15.141:0/884869526 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f4938153e50 0x7f493815a7c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4938089670 tx=0x7f4934009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=188f9095907ca68f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:41.856 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493ca85700 1 -- 172.21.15.141:0/884869526 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400b640 con 0x7f4938153e50 2023-06-23T00:35:41.856 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493ca85700 1 -- 172.21.15.141:0/884869526 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493400b7a0 con 0x7f4938153e50 2023-06-23T00:35:41.856 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493ca85700 1 -- 172.21.15.141:0/884869526 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400bb40 con 0x7f4938153e50 2023-06-23T00:35:41.857 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493fceb700 1 -- 172.21.15.141:0/884869526 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f4938153e50 msgr2=0x7f493815a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:41.857 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493fceb700 1 --2- 172.21.15.141:0/884869526 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f4938153e50 0x7f493815a7c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4938089670 tx=0x7f4934009250 comp rx=0 tx=0).stop 2023-06-23T00:35:41.857 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493fceb700 1 -- 172.21.15.141:0/884869526 shutdown_connections 2023-06-23T00:35:41.857 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493fceb700 1 --2- 172.21.15.141:0/884869526 >> v2:172.21.15.55:3300/0 conn(0x7f4938153a80 0x7f49380d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.857 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493fceb700 1 --2- 172.21.15.141:0/884869526 >> v2:172.21.15.141:3300/0 conn(0x7f49380b2010 0x7f49380b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.857 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493fceb700 1 --2- 172.21.15.141:0/884869526 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f4938153e50 0x7f493815a7c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.857 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493fceb700 1 -- 172.21.15.141:0/884869526 >> 172.21.15.141:0/884869526 conn(0x7f4938070290 msgr2=0x7f4938070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:41.858 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493fceb700 1 -- 172.21.15.141:0/884869526 shutdown_connections 2023-06-23T00:35:41.858 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.850+0000 7f493fceb700 1 -- 172.21.15.141:0/884869526 wait complete. 2023-06-23T00:35:41.860 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.851+0000 7f493fceb700 1 Processor -- start 2023-06-23T00:35:41.860 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.851+0000 7f493fceb700 1 -- start start 2023-06-23T00:35:41.860 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493fceb700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f49380b2010 0x7f4938159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.860 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493fceb700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4938153a80 0x7f49381504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.860 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493fceb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4938153e50 0x7f493813df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:41.861 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493fceb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493815a3f0 con 0x7f4938153e50 2023-06-23T00:35:41.861 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493fceb700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493815a6d0 con 0x7f4938153a80 2023-06-23T00:35:41.861 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493fceb700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493815a9b0 con 0x7f49380b2010 2023-06-23T00:35:41.861 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493da87700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f49380b2010 0x7f4938159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.861 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493da87700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f49380b2010 0x7f4938159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3301/0 says I am v2:172.21.15.141:50330/0 (socket says 172.21.15.141:50330) 2023-06-23T00:35:41.861 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493d286700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4938153a80 0x7f49381504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.861 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493da87700 1 -- 172.21.15.141:0/9790546 learned_addr learned my addr 172.21.15.141:0/9790546 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:41.862 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.852+0000 7f493e288700 1 --2- 172.21.15.141:0/9790546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4938153e50 0x7f493813df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:41.862 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f493da87700 1 -- 172.21.15.141:0/9790546 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4938153a80 msgr2=0x7f49381504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:41.862 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f493da87700 1 --2- 172.21.15.141:0/9790546 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4938153a80 0x7f49381504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.862 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f493da87700 1 -- 172.21.15.141:0/9790546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4938153e50 msgr2=0x7f493813df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:41.862 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f493da87700 1 --2- 172.21.15.141:0/9790546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4938153e50 0x7f493813df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.862 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f493da87700 1 -- 172.21.15.141:0/9790546 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930009a20 con 0x7f49380b2010 2023-06-23T00:35:41.862 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f493d286700 1 --2- 172.21.15.141:0/9790546 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4938153a80 0x7f49381504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:41.863 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f493da87700 1 --2- 172.21.15.141:0/9790546 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f49380b2010 0x7f4938159d60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f492800ec80 tx=0x7f49280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:41.863 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f492dffb700 1 -- 172.21.15.141:0/9790546 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800c850 con 0x7f49380b2010 2023-06-23T00:35:41.863 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f492dffb700 1 -- 172.21.15.141:0/9790546 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492800c9b0 con 0x7f49380b2010 2023-06-23T00:35:41.863 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f493fceb700 1 -- 172.21.15.141:0/9790546 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49340094f0 con 0x7f49380b2010 2023-06-23T00:35:41.863 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f492dffb700 1 -- 172.21.15.141:0/9790546 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800cd50 con 0x7f49380b2010 2023-06-23T00:35:41.863 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f493fceb700 1 -- 172.21.15.141:0/9790546 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493814c2e0 con 0x7f49380b2010 2023-06-23T00:35:41.863 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f492dffb700 1 -- 172.21.15.141:0/9790546 <== mon.2 v2:172.21.15.141:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4928005c80 con 0x7f49380b2010 2023-06-23T00:35:41.864 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.853+0000 7f492dffb700 1 -- 172.21.15.141:0/9790546 <== mon.2 v2:172.21.15.141:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f4928006020 con 0x7f49380b2010 2023-06-23T00:35:41.864 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.854+0000 7f493fceb700 1 -- 172.21.15.141:0/9790546 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493804c1b0 con 0x7f49380b2010 2023-06-23T00:35:41.864 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.858+0000 7f492dffb700 1 -- 172.21.15.141:0/9790546 <== mon.2 v2:172.21.15.141:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f4928021060 con 0x7f49380b2010 2023-06-23T00:35:41.944 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.941+0000 7f493fceb700 1 -- 172.21.15.141:0/9790546 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "osd new", "uuid": "518b1bc8-fb1c-405f-8346-fda7978829a6", "id": 1} v 0) v1 -- 0x7f49380d0c20 con 0x7f49380b2010 2023-06-23T00:35:41.947 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.944+0000 7f492dffb700 1 -- 172.21.15.141:0/9790546 <== mon.2 v2:172.21.15.141:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "518b1bc8-fb1c-405f-8346-fda7978829a6", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4928017070 con 0x7f49380b2010 2023-06-23T00:35:41.948 INFO:teuthology.orchestra.run.smithi141.stdout:1 2023-06-23T00:35:41.948 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.945+0000 7f493fceb700 1 -- 172.21.15.141:0/9790546 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f49380b2010 msgr2=0x7f4938159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:41.949 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.945+0000 7f493fceb700 1 --2- 172.21.15.141:0/9790546 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f49380b2010 0x7f4938159d60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f492800ec80 tx=0x7f49280091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.949 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.945+0000 7f493fceb700 1 -- 172.21.15.141:0/9790546 shutdown_connections 2023-06-23T00:35:41.949 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.945+0000 7f493fceb700 1 --2- 172.21.15.141:0/9790546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4938153e50 0x7f493813df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.949 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.945+0000 7f493fceb700 1 --2- 172.21.15.141:0/9790546 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f49380b2010 0x7f4938159d60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.949 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.945+0000 7f493fceb700 1 --2- 172.21.15.141:0/9790546 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f4938153a80 0x7f49381504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:41.949 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.945+0000 7f493fceb700 1 -- 172.21.15.141:0/9790546 >> 172.21.15.141:0/9790546 conn(0x7f4938070290 msgr2=0x7f49380b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:41.950 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.945+0000 7f493fceb700 1 -- 172.21.15.141:0/9790546 shutdown_connections 2023-06-23T00:35:41.950 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:41.945+0000 7f493fceb700 1 -- 172.21.15.141:0/9790546 wait complete. 2023-06-23T00:35:41.961 DEBUG:teuthology.orchestra.run.smithi141:> sudo ceph --cluster ceph osd new 636a04f1-2379-43ca-9f02-0ee33d2af90a 2 2023-06-23T00:35:42.079 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.076+0000 7fe5c16dd700 1 Processor -- start 2023-06-23T00:35:42.079 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.076+0000 7fe5c16dd700 1 -- start start 2023-06-23T00:35:42.081 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.077+0000 7fe5c16dd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe5bc0b23d0 0x7fe5bc1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.081 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.077+0000 7fe5c16dd700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5bc0cb7d0 con 0x7fe5bc159310 2023-06-23T00:35:42.081 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.077+0000 7fe5c16dd700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5bc0c72b0 con 0x7fe5bc0b00e0 2023-06-23T00:35:42.082 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.077+0000 7fe5c16dd700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5bc0b2e10 con 0x7fe5bc0b23d0 2023-06-23T00:35:42.082 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.077+0000 7fe5c0edc700 1 --1- >> v1:172.21.15.141:6789/0 conn(0x7fe5bc0b00e0 0x7fe5bc0b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.141:6789/0 says I am v1:172.21.15.141:59448/0 (socket says 172.21.15.141:59448) 2023-06-23T00:35:42.082 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.077+0000 7fe5c0edc700 1 -- 172.21.15.141:0/3496698857 learned_addr learned my addr 172.21.15.141:0/3496698857 (peer_addr_for_me v1:172.21.15.141:0/0) 2023-06-23T00:35:42.082 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.077+0000 7fe5bb7fe700 1 --2- 172.21.15.141:0/3496698857 >> v2:172.21.15.55:3300/0 conn(0x7fe5bc0b23d0 0x7fe5bc1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.082 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.077+0000 7fe5baffd700 1 -- 172.21.15.141:0/3496698857 <== mon.1 v1:172.21.15.141:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148364390 0 0) 0x7fe5bc0c72b0 con 0x7fe5bc0b00e0 2023-06-23T00:35:42.082 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.077+0000 7fe5baffd700 1 -- 172.21.15.141:0/3496698857 --> v1:172.21.15.141:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5a4003570 con 0x7fe5bc0b00e0 2023-06-23T00:35:42.082 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.078+0000 7fe5bb7fe700 1 -- 172.21.15.141:0/3496698857 >> v1:172.21.15.141:6789/0 conn(0x7fe5bc0b00e0 legacy=0x7fe5bc0b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.083 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.078+0000 7fe5bb7fe700 1 -- 172.21.15.141:0/3496698857 >> v1:172.21.15.55:6789/0 conn(0x7fe5bc159310 legacy=0x7fe5bc15dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.083 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.078+0000 7fe5baffd700 1 -- 172.21.15.141:0/3496698857 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049543282 0 0) 0x7fe5bc0cb7d0 con 0x7fe5bc159310 2023-06-23T00:35:42.083 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.078+0000 7fe5bb7fe700 1 -- 172.21.15.141:0/3496698857 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc0b2850 con 0x7fe5bc0b23d0 2023-06-23T00:35:42.083 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.078+0000 7fe5bb7fe700 1 --2- 172.21.15.141:0/3496698857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0b23d0 0x7fe5bc1596e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0098e0 tx=0x7fe5ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487b35de802fbec3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:42.083 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.078+0000 7fe5baffd700 1 -- 172.21.15.141:0/3496698857 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac00b640 con 0x7fe5bc0b23d0 2023-06-23T00:35:42.083 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.078+0000 7fe5baffd700 1 -- 172.21.15.141:0/3496698857 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ac00b7a0 con 0x7fe5bc0b23d0 2023-06-23T00:35:42.083 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.078+0000 7fe5baffd700 1 -- 172.21.15.141:0/3496698857 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac00bb40 con 0x7fe5bc0b23d0 2023-06-23T00:35:42.086 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.078+0000 7fe5c16dd700 1 -- 172.21.15.141:0/3496698857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0b23d0 msgr2=0x7fe5bc1596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.086 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.078+0000 7fe5c16dd700 1 --2- 172.21.15.141:0/3496698857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0b23d0 0x7fe5bc1596e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0098e0 tx=0x7fe5ac0091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.086 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.079+0000 7fe5c16dd700 1 -- 172.21.15.141:0/3496698857 shutdown_connections 2023-06-23T00:35:42.086 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.079+0000 7fe5c16dd700 1 --2- 172.21.15.141:0/3496698857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0b23d0 0x7fe5bc1596e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.087 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.079+0000 7fe5c16dd700 1 -- 172.21.15.141:0/3496698857 >> 172.21.15.141:0/3496698857 conn(0x7fe5bc070290 msgr2=0x7fe5bc070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:42.087 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.079+0000 7fe5c16dd700 1 -- 172.21.15.141:0/3496698857 shutdown_connections 2023-06-23T00:35:42.087 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.079+0000 7fe5c16dd700 1 -- 172.21.15.141:0/3496698857 wait complete. 2023-06-23T00:35:42.087 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.080+0000 7fe5c16dd700 1 Processor -- start 2023-06-23T00:35:42.087 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.080+0000 7fe5c16dd700 1 -- start start 2023-06-23T00:35:42.087 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c16dd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0b00e0 0x7fe5bc16a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.087 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c16dd700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe5bc159310 0x7fe5bc157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.087 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c16dd700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe5bc0c7510 0x7fe5bc14f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.088 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c16dd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc1578b0 con 0x7fe5bc0b00e0 2023-06-23T00:35:42.088 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c16dd700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc1559d0 con 0x7fe5bc159310 2023-06-23T00:35:42.088 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c16dd700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc157ae0 con 0x7fe5bc0c7510 2023-06-23T00:35:42.088 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5bb7fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe5bc159310 0x7fe5bc157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.088 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c0edc700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe5bc0c7510 0x7fe5bc14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.088 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c0edc700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe5bc0c7510 0x7fe5bc14f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3301/0 says I am v2:172.21.15.141:50336/0 (socket says 172.21.15.141:50336) 2023-06-23T00:35:42.088 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5bb7fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe5bc159310 0x7fe5bc157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.141:45730/0 (socket says 172.21.15.141:45730) 2023-06-23T00:35:42.089 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c0edc700 1 -- 172.21.15.141:0/809191369 learned_addr learned my addr 172.21.15.141:0/809191369 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:42.089 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5bbfff700 1 --2- 172.21.15.141:0/809191369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0b00e0 0x7fe5bc16a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.089 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5bb7fe700 1 -- 172.21.15.141:0/809191369 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe5bc0c7510 msgr2=0x7fe5bc14f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.089 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5bb7fe700 1 --2- 172.21.15.141:0/809191369 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe5bc0c7510 0x7fe5bc14f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.089 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5bb7fe700 1 -- 172.21.15.141:0/809191369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0b00e0 msgr2=0x7fe5bc16a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.089 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5bb7fe700 1 --2- 172.21.15.141:0/809191369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0b00e0 0x7fe5bc16a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.089 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5bb7fe700 1 -- 172.21.15.141:0/809191369 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac009420 con 0x7fe5bc159310 2023-06-23T00:35:42.089 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c0edc700 1 --2- 172.21.15.141:0/809191369 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe5bc0c7510 0x7fe5bc14f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:42.090 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5bb7fe700 1 --2- 172.21.15.141:0/809191369 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe5bc159310 0x7fe5bc157370 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac016000 tx=0x7fe5ac017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:42.090 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5a3fff700 1 -- 172.21.15.141:0/809191369 <== mon.1 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac01f070 con 0x7fe5bc159310 2023-06-23T00:35:42.090 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5a3fff700 1 -- 172.21.15.141:0/809191369 <== mon.1 v2:172.21.15.141:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ac002f70 con 0x7fe5bc159310 2023-06-23T00:35:42.090 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c16dd700 1 -- 172.21.15.141:0/809191369 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5bc155c70 con 0x7fe5bc159310 2023-06-23T00:35:42.090 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5a3fff700 1 -- 172.21.15.141:0/809191369 <== mon.1 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac00b910 con 0x7fe5bc159310 2023-06-23T00:35:42.090 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.081+0000 7fe5c16dd700 1 -- 172.21.15.141:0/809191369 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5bc14fe60 con 0x7fe5bc159310 2023-06-23T00:35:42.090 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.082+0000 7fe5a3fff700 1 -- 172.21.15.141:0/809191369 <== mon.1 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe5ac023070 con 0x7fe5bc159310 2023-06-23T00:35:42.090 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.082+0000 7fe5a3fff700 1 -- 172.21.15.141:0/809191369 <== mon.1 v2:172.21.15.141:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fe5ac00ba70 con 0x7fe5bc159310 2023-06-23T00:35:42.091 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.082+0000 7fe5c16dd700 1 -- 172.21.15.141:0/809191369 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5bc067f00 con 0x7fe5bc159310 2023-06-23T00:35:42.091 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.084+0000 7fe5a3fff700 1 -- 172.21.15.141:0/809191369 <== mon.1 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fe5ac02a070 con 0x7fe5bc159310 2023-06-23T00:35:42.114 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:42.111+0000 7fa57af57180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T00:35:42.156 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:42.152+0000 7fde1d809180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T00:35:42.170 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.167+0000 7fe5c16dd700 1 -- 172.21.15.141:0/809191369 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "osd new", "uuid": "636a04f1-2379-43ca-9f02-0ee33d2af90a", "id": 2} v 0) v1 -- 0x7fe5bc156190 con 0x7fe5bc159310 2023-06-23T00:35:42.175 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.172+0000 7fe5a3fff700 1 -- 172.21.15.141:0/809191369 <== mon.1 v2:172.21.15.141:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "636a04f1-2379-43ca-9f02-0ee33d2af90a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe5bc156190 con 0x7fe5bc159310 2023-06-23T00:35:42.176 INFO:teuthology.orchestra.run.smithi141.stdout:2 2023-06-23T00:35:42.177 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.173+0000 7fe5c16dd700 1 -- 172.21.15.141:0/809191369 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe5bc159310 msgr2=0x7fe5bc157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.177 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.173+0000 7fe5c16dd700 1 --2- 172.21.15.141:0/809191369 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe5bc159310 0x7fe5bc157370 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac016000 tx=0x7fe5ac017000 comp rx=0 tx=0).stop 2023-06-23T00:35:42.177 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.173+0000 7fe5c16dd700 1 -- 172.21.15.141:0/809191369 shutdown_connections 2023-06-23T00:35:42.178 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.173+0000 7fe5c16dd700 1 --2- 172.21.15.141:0/809191369 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe5bc0c7510 0x7fe5bc14f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.178 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.173+0000 7fe5c16dd700 1 --2- 172.21.15.141:0/809191369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5bc0b00e0 0x7fe5bc16a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.178 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.173+0000 7fe5c16dd700 1 --2- 172.21.15.141:0/809191369 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe5bc159310 0x7fe5bc157370 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.179 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.173+0000 7fe5c16dd700 1 -- 172.21.15.141:0/809191369 >> 172.21.15.141:0/809191369 conn(0x7fe5bc070290 msgr2=0x7fe5bc0b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:42.180 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.173+0000 7fe5c16dd700 1 -- 172.21.15.141:0/809191369 shutdown_connections 2023-06-23T00:35:42.180 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.173+0000 7fe5c16dd700 1 -- 172.21.15.141:0/809191369 wait complete. 2023-06-23T00:35:42.188 DEBUG:teuthology.orchestra.run.smithi141:> sudo ceph --cluster ceph osd new b893aff1-9f4d-42ca-9af6-42fd6ba3d92c 3 2023-06-23T00:35:42.252 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:42.249+0000 7fa57af57180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T00:35:42.294 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:42.290+0000 7fde1d809180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T00:35:42.303 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.300+0000 7fc7389bd700 1 Processor -- start 2023-06-23T00:35:42.303 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.300+0000 7fc7389bd700 1 -- start start 2023-06-23T00:35:42.305 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc7389bd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc734159e10 0x7fc734069f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.305 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc7389bd700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7fc73406a4a0 0x7fc7340c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.305 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc7389bd700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7340c7a20 con 0x7fc734159a40 2023-06-23T00:35:42.305 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc7389bd700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73406a8e0 con 0x7fc734159e10 2023-06-23T00:35:42.305 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc7389bd700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7340c9e50 con 0x7fc73406a4a0 2023-06-23T00:35:42.305 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc732d9d700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7fc73406a4a0 0x7fc7340c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.306 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc732d9d700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7fc73406a4a0 0x7fc7340c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.141:45734/0 (socket says 172.21.15.141:45734) 2023-06-23T00:35:42.306 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc732d9d700 1 -- 172.21.15.141:0/2698369619 learned_addr learned my addr 172.21.15.141:0/2698369619 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:42.306 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc731d9b700 1 --2- 172.21.15.141:0/2698369619 >> v2:172.21.15.55:3300/0 conn(0x7fc734159e10 0x7fc734069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.306 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc732d9d700 1 -- 172.21.15.141:0/2698369619 >> v2:172.21.15.55:3300/0 conn(0x7fc734159e10 msgr2=0x7fc734069f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:42.306 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc732d9d700 1 --2- 172.21.15.141:0/2698369619 >> v2:172.21.15.55:3300/0 conn(0x7fc734159e10 0x7fc734069f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.306 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc732d9d700 1 -- 172.21.15.141:0/2698369619 >> v1:172.21.15.55:6789/0 conn(0x7fc734159a40 legacy=0x7fc7340b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.306 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc732d9d700 1 -- 172.21.15.141:0/2698369619 --> v2:172.21.15.141:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7340b0c10 con 0x7fc73406a4a0 2023-06-23T00:35:42.307 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.301+0000 7fc73159a700 1 -- 172.21.15.141:0/2698369619 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889477099 0 0) 0x7fc7340c7a20 con 0x7fc734159a40 2023-06-23T00:35:42.307 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.302+0000 7fc732d9d700 1 --2- 172.21.15.141:0/2698369619 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc73406a4a0 0x7fc7340c7eb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc73409b060 tx=0x7fc72c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1d0b673d3b52f2e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:42.307 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.302+0000 7fc73159a700 1 -- 172.21.15.141:0/2698369619 <== mon.1 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00e070 con 0x7fc73406a4a0 2023-06-23T00:35:42.307 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.302+0000 7fc73159a700 1 -- 172.21.15.141:0/2698369619 <== mon.1 v2:172.21.15.141:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c00ba20 con 0x7fc73406a4a0 2023-06-23T00:35:42.307 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.302+0000 7fc73159a700 1 -- 172.21.15.141:0/2698369619 <== mon.1 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c0092e0 con 0x7fc73406a4a0 2023-06-23T00:35:42.310 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.302+0000 7fc7389bd700 1 -- 172.21.15.141:0/2698369619 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc73406a4a0 msgr2=0x7fc7340c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.310 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.302+0000 7fc7389bd700 1 --2- 172.21.15.141:0/2698369619 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc73406a4a0 0x7fc7340c7eb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc73409b060 tx=0x7fc72c009840 comp rx=0 tx=0).stop 2023-06-23T00:35:42.310 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.303+0000 7fc7389bd700 1 -- 172.21.15.141:0/2698369619 shutdown_connections 2023-06-23T00:35:42.311 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.303+0000 7fc7389bd700 1 --2- 172.21.15.141:0/2698369619 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc73406a4a0 0x7fc7340c7eb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.311 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.303+0000 7fc7389bd700 1 --2- 172.21.15.141:0/2698369619 >> v2:172.21.15.55:3300/0 conn(0x7fc734159e10 0x7fc734069f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.311 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.303+0000 7fc7389bd700 1 -- 172.21.15.141:0/2698369619 >> 172.21.15.141:0/2698369619 conn(0x7fc734068520 msgr2=0x7fc734068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:42.311 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.303+0000 7fc7389bd700 1 -- 172.21.15.141:0/2698369619 shutdown_connections 2023-06-23T00:35:42.311 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.303+0000 7fc7389bd700 1 -- 172.21.15.141:0/2698369619 wait complete. 2023-06-23T00:35:42.311 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc7389bd700 1 Processor -- start 2023-06-23T00:35:42.311 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc7389bd700 1 -- start start 2023-06-23T00:35:42.312 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc7389bd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73406a4a0 0x7fc734158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.312 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc7389bd700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fc734159a40 0x7fc7340abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.312 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc7389bd700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc734159e10 0x7fc7340ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.312 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc7389bd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734169980 con 0x7fc73406a4a0 2023-06-23T00:35:42.312 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc7389bd700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734158b50 con 0x7fc734159e10 2023-06-23T00:35:42.312 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc7389bd700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340ab580 con 0x7fc734159a40 2023-06-23T00:35:42.312 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc732d9d700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc734159e10 0x7fc7340ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.312 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc731d9b700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fc734159a40 0x7fc7340abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.313 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc732d9d700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc734159e10 0x7fc7340ac9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.141:45750/0 (socket says 172.21.15.141:45750) 2023-06-23T00:35:42.313 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc732d9d700 1 -- 172.21.15.141:0/1738883729 learned_addr learned my addr 172.21.15.141:0/1738883729 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:42.313 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.304+0000 7fc73259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73406a4a0 0x7fc734158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.313 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc732d9d700 1 -- 172.21.15.141:0/1738883729 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fc734159a40 msgr2=0x7fc7340abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.313 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc732d9d700 1 --2- 172.21.15.141:0/1738883729 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fc734159a40 0x7fc7340abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.313 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc732d9d700 1 -- 172.21.15.141:0/1738883729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73406a4a0 msgr2=0x7fc734158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.313 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc732d9d700 1 --2- 172.21.15.141:0/1738883729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73406a4a0 0x7fc734158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.314 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc732d9d700 1 -- 172.21.15.141:0/1738883729 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c009ae0 con 0x7fc734159e10 2023-06-23T00:35:42.314 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc73259c700 1 --2- 172.21.15.141:0/1738883729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73406a4a0 0x7fc734158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:42.314 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc732d9d700 1 --2- 172.21.15.141:0/1738883729 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc734159e10 0x7fc7340ac9c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009ab0 tx=0x7fc72c018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:42.314 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc7227fc700 1 -- 172.21.15.141:0/1738883729 <== mon.1 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c008ac0 con 0x7fc734159e10 2023-06-23T00:35:42.314 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc7227fc700 1 -- 172.21.15.141:0/1738883729 <== mon.1 v2:172.21.15.141:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c008c20 con 0x7fc734159e10 2023-06-23T00:35:42.314 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc7389bd700 1 -- 172.21.15.141:0/1738883729 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7340ab860 con 0x7fc734159e10 2023-06-23T00:35:42.314 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc7227fc700 1 -- 172.21.15.141:0/1738883729 <== mon.1 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c026460 con 0x7fc734159e10 2023-06-23T00:35:42.314 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc7389bd700 1 -- 172.21.15.141:0/1738883729 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7340c53d0 con 0x7fc734159e10 2023-06-23T00:35:42.315 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc7227fc700 1 -- 172.21.15.141:0/1738883729 <== mon.1 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc72c003040 con 0x7fc734159e10 2023-06-23T00:35:42.315 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.305+0000 7fc7227fc700 1 -- 172.21.15.141:0/1738883729 <== mon.1 v2:172.21.15.141:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fc72c026c50 con 0x7fc734159e10 2023-06-23T00:35:42.315 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.306+0000 7fc7389bd700 1 -- 172.21.15.141:0/1738883729 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7340c5a20 con 0x7fc734159e10 2023-06-23T00:35:42.315 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.308+0000 7fc7227fc700 1 -- 172.21.15.141:0/1738883729 <== mon.1 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fc72c02b020 con 0x7fc734159e10 2023-06-23T00:35:42.332 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:42.329+0000 7fa57af57180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T00:35:42.373 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:42.369+0000 7fde1d809180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T00:35:42.394 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.391+0000 7fc7389bd700 1 -- 172.21.15.141:0/1738883729 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b893aff1-9f4d-42ca-9af6-42fd6ba3d92c", "id": 3} v 0) v1 -- 0x7fc73404c1b0 con 0x7fc734159e10 2023-06-23T00:35:42.401 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.398+0000 7fc7227fc700 1 -- 172.21.15.141:0/1738883729 <== mon.1 v2:172.21.15.141:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b893aff1-9f4d-42ca-9af6-42fd6ba3d92c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc72c021070 con 0x7fc734159e10 2023-06-23T00:35:42.402 INFO:teuthology.orchestra.run.smithi141.stdout:3 2023-06-23T00:35:42.403 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.400+0000 7fc7389bd700 1 -- 172.21.15.141:0/1738883729 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc734159e10 msgr2=0x7fc7340ac9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.403 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.400+0000 7fc7389bd700 1 --2- 172.21.15.141:0/1738883729 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc734159e10 0x7fc7340ac9c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009ab0 tx=0x7fc72c018000 comp rx=0 tx=0).stop 2023-06-23T00:35:42.404 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.400+0000 7fc7389bd700 1 -- 172.21.15.141:0/1738883729 shutdown_connections 2023-06-23T00:35:42.404 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.400+0000 7fc7389bd700 1 --2- 172.21.15.141:0/1738883729 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fc734159e10 0x7fc7340ac9c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.404 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.400+0000 7fc7389bd700 1 --2- 172.21.15.141:0/1738883729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73406a4a0 0x7fc734158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.404 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.400+0000 7fc7389bd700 1 --2- 172.21.15.141:0/1738883729 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fc734159a40 0x7fc7340abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.404 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.400+0000 7fc7389bd700 1 -- 172.21.15.141:0/1738883729 >> 172.21.15.141:0/1738883729 conn(0x7fc734068520 msgr2=0x7fc7340c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:42.404 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.400+0000 7fc7389bd700 1 -- 172.21.15.141:0/1738883729 shutdown_connections 2023-06-23T00:35:42.404 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.400+0000 7fc7389bd700 1 -- 172.21.15.141:0/1738883729 wait complete. 2023-06-23T00:35:42.419 DEBUG:teuthology.orchestra.run.smithi141:> sudo ceph --cluster ceph osd new 259d9777-c081-4c0b-9d83-46d348872e01 4 2023-06-23T00:35:42.530 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:42.527+0000 7fa57af57180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T00:35:42.535 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.533+0000 7f46adaca700 1 Processor -- start 2023-06-23T00:35:42.536 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.533+0000 7f46adaca700 1 -- start start 2023-06-23T00:35:42.537 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.533+0000 7f46adaca700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46a81572d0 0x7f46a815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.537 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.533+0000 7f46adaca700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f46a80b1080 0x7f46a80ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.538 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.533+0000 7f46adaca700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a815bff0 con 0x7f46a81576a0 2023-06-23T00:35:42.538 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.533+0000 7f46adaca700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a80ce4f0 con 0x7f46a81572d0 2023-06-23T00:35:42.538 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.533+0000 7f46adaca700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a80b2190 con 0x7f46a80b1080 2023-06-23T00:35:42.538 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46a7fff700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f46a80b1080 0x7f46a80ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.538 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46ad2c9700 1 --1- >> v1:172.21.15.141:6789/0 conn(0x7f46a81576a0 0x7f46a806c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.141:6789/0 says I am v1:172.21.15.141:59450/0 (socket says 172.21.15.141:59450) 2023-06-23T00:35:42.538 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46ad2c9700 1 -- 172.21.15.141:0/1405738581 learned_addr learned my addr 172.21.15.141:0/1405738581 (peer_addr_for_me v1:172.21.15.141:0/0) 2023-06-23T00:35:42.538 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46acac8700 1 --2- 172.21.15.141:0/1405738581 >> v2:172.21.15.55:3300/0 conn(0x7f46a81572d0 0x7f46a815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.538 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46a7fff700 1 -- 172.21.15.141:0/1405738581 >> v2:172.21.15.55:3300/0 conn(0x7f46a81572d0 msgr2=0x7f46a815bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:42.539 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46a7fff700 1 --2- 172.21.15.141:0/1405738581 >> v2:172.21.15.55:3300/0 conn(0x7f46a81572d0 0x7f46a815bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.539 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46a7fff700 1 -- 172.21.15.141:0/1405738581 >> v1:172.21.15.141:6789/0 conn(0x7f46a81576a0 legacy=0x7f46a806c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.539 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46a7fff700 1 -- 172.21.15.141:0/1405738581 --> v2:172.21.15.141:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a80b1580 con 0x7f46a80b1080 2023-06-23T00:35:42.539 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46a77fe700 1 -- 172.21.15.141:0/1405738581 <== mon.1 v1:172.21.15.141:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928924269 0 0) 0x7f46a0001490 con 0x7f46a81576a0 2023-06-23T00:35:42.539 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46acac8700 1 --2- 172.21.15.141:0/1405738581 >> v2:172.21.15.55:3300/0 conn(0x7f46a81572d0 0x7f46a815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T00:35:42.539 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46a7fff700 1 --2- 172.21.15.141:0/1405738581 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 0x7f46a80ca1f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f46a806e820 tx=0x7f46940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4929d1098c38e45 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:42.539 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46a77fe700 1 -- 172.21.15.141:0/1405738581 <== mon.1 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469400b6b0 con 0x7f46a80b1080 2023-06-23T00:35:42.540 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46a77fe700 1 -- 172.21.15.141:0/1405738581 <== mon.1 v2:172.21.15.141:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469400b810 con 0x7f46a80b1080 2023-06-23T00:35:42.540 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.534+0000 7f46a77fe700 1 -- 172.21.15.141:0/1405738581 <== mon.1 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469400bbb0 con 0x7f46a80b1080 2023-06-23T00:35:42.542 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.535+0000 7f46adaca700 1 -- 172.21.15.141:0/1405738581 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 msgr2=0x7f46a80ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.542 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.535+0000 7f46adaca700 1 --2- 172.21.15.141:0/1405738581 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 0x7f46a80ca1f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f46a806e820 tx=0x7f46940091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.542 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.535+0000 7f46adaca700 1 -- 172.21.15.141:0/1405738581 shutdown_connections 2023-06-23T00:35:42.542 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.535+0000 7f46adaca700 1 --2- 172.21.15.141:0/1405738581 >> v2:172.21.15.55:3300/0 conn(0x7f46a81572d0 0x7f46a815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.543 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.535+0000 7f46adaca700 1 --2- 172.21.15.141:0/1405738581 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 0x7f46a80ca1f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.543 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.535+0000 7f46adaca700 1 -- 172.21.15.141:0/1405738581 >> 172.21.15.141:0/1405738581 conn(0x7f46a806b1c0 msgr2=0x7f46a806b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:42.543 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.535+0000 7f46adaca700 1 -- 172.21.15.141:0/1405738581 shutdown_connections 2023-06-23T00:35:42.543 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.535+0000 7f46adaca700 1 -- 172.21.15.141:0/1405738581 wait complete. 2023-06-23T00:35:42.543 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.536+0000 7f46adaca700 1 Processor -- start 2023-06-23T00:35:42.543 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.536+0000 7f46adaca700 1 -- start start 2023-06-23T00:35:42.543 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46adaca700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 0x7f46a8145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.544 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46adaca700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f46a81572d0 0x7f46a81598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.544 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46adaca700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a81576a0 0x7f46a8159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.544 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46adaca700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a816c4c0 con 0x7f46a81576a0 2023-06-23T00:35:42.544 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46adaca700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80c3850 con 0x7f46a80b1080 2023-06-23T00:35:42.544 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46adaca700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a816c1e0 con 0x7f46a81572d0 2023-06-23T00:35:42.544 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46acac8700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 0x7f46a8145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.544 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46a7fff700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f46a81572d0 0x7f46a81598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.545 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46acac8700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 0x7f46a8145cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.141:45764/0 (socket says 172.21.15.141:45764) 2023-06-23T00:35:42.545 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46acac8700 1 -- 172.21.15.141:0/277576347 learned_addr learned my addr 172.21.15.141:0/277576347 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:42.545 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46ad2c9700 1 --2- 172.21.15.141:0/277576347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a81576a0 0x7f46a8159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.545 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46a7fff700 1 -- 172.21.15.141:0/277576347 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 msgr2=0x7f46a8145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.545 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46a7fff700 1 --2- 172.21.15.141:0/277576347 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 0x7f46a8145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.545 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46a7fff700 1 -- 172.21.15.141:0/277576347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a81576a0 msgr2=0x7f46a8159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.545 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46a7fff700 1 --2- 172.21.15.141:0/277576347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a81576a0 0x7f46a8159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.545 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46a7fff700 1 -- 172.21.15.141:0/277576347 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694009420 con 0x7f46a81572d0 2023-06-23T00:35:42.546 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46acac8700 1 --2- 172.21.15.141:0/277576347 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 0x7f46a8145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:42.546 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.537+0000 7f46a7fff700 1 --2- 172.21.15.141:0/277576347 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f46a81572d0 0x7f46a81598f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4694016000 tx=0x7f4694017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:42.546 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.538+0000 7f46a4ff9700 1 -- 172.21.15.141:0/277576347 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694000d10 con 0x7f46a81572d0 2023-06-23T00:35:42.546 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.538+0000 7f46a4ff9700 1 -- 172.21.15.141:0/277576347 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469400bb90 con 0x7f46a81572d0 2023-06-23T00:35:42.546 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.538+0000 7f46adaca700 1 -- 172.21.15.141:0/277576347 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a80c3200 con 0x7f46a81572d0 2023-06-23T00:35:42.546 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.538+0000 7f46a4ff9700 1 -- 172.21.15.141:0/277576347 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694011650 con 0x7f46a81572d0 2023-06-23T00:35:42.546 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.538+0000 7f46adaca700 1 -- 172.21.15.141:0/277576347 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a8146210 con 0x7f46a81572d0 2023-06-23T00:35:42.546 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.538+0000 7f46a4ff9700 1 -- 172.21.15.141:0/277576347 <== mon.2 v2:172.21.15.141:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4694011cc0 con 0x7f46a81572d0 2023-06-23T00:35:42.547 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.538+0000 7f46a4ff9700 1 -- 172.21.15.141:0/277576347 <== mon.2 v2:172.21.15.141:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f4694025900 con 0x7f46a81572d0 2023-06-23T00:35:42.547 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.539+0000 7f46adaca700 1 -- 172.21.15.141:0/277576347 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a80c3b30 con 0x7f46a81572d0 2023-06-23T00:35:42.547 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.541+0000 7f46a4ff9700 1 -- 172.21.15.141:0/277576347 <== mon.2 v2:172.21.15.141:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f4694015020 con 0x7f46a81572d0 2023-06-23T00:35:42.565 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:42.561+0000 7fde1d809180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T00:35:42.674 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.671+0000 7f46adaca700 1 -- 172.21.15.141:0/277576347 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "osd new", "uuid": "259d9777-c081-4c0b-9d83-46d348872e01", "id": 4} v 0) v1 -- 0x7f46a80c3060 con 0x7f46a81572d0 2023-06-23T00:35:42.681 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.678+0000 7f46a4ff9700 1 -- 172.21.15.141:0/277576347 <== mon.2 v2:172.21.15.141:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "259d9777-c081-4c0b-9d83-46d348872e01", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f46a80c3060 con 0x7f46a81572d0 2023-06-23T00:35:42.681 INFO:teuthology.orchestra.run.smithi141.stdout:4 2023-06-23T00:35:42.682 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.679+0000 7f46adaca700 1 -- 172.21.15.141:0/277576347 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f46a81572d0 msgr2=0x7f46a81598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.683 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.679+0000 7f46adaca700 1 --2- 172.21.15.141:0/277576347 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f46a81572d0 0x7f46a81598f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4694016000 tx=0x7f4694017000 comp rx=0 tx=0).stop 2023-06-23T00:35:42.683 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.679+0000 7f46adaca700 1 -- 172.21.15.141:0/277576347 shutdown_connections 2023-06-23T00:35:42.683 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.679+0000 7f46adaca700 1 --2- 172.21.15.141:0/277576347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46a81576a0 0x7f46a8159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.683 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.679+0000 7f46adaca700 1 --2- 172.21.15.141:0/277576347 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f46a80b1080 0x7f46a8145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.683 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.679+0000 7f46adaca700 1 --2- 172.21.15.141:0/277576347 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f46a81572d0 0x7f46a81598f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.683 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.679+0000 7f46adaca700 1 -- 172.21.15.141:0/277576347 >> 172.21.15.141:0/277576347 conn(0x7f46a806b1c0 msgr2=0x7f46a80cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:42.684 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.679+0000 7f46adaca700 1 -- 172.21.15.141:0/277576347 shutdown_connections 2023-06-23T00:35:42.684 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.679+0000 7f46adaca700 1 -- 172.21.15.141:0/277576347 wait complete. 2023-06-23T00:35:42.694 DEBUG:teuthology.orchestra.run.smithi141:> sudo ceph --cluster ceph osd new 56cfcc58-3f4d-4637-ae5d-a2a52dbdc37f 5 2023-06-23T00:35:42.746 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:42.743+0000 7fa57af57180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T00:35:42.747 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:42.742+0000 7fde1d809180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T00:35:42.809 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.806+0000 7f18d46e7700 1 Processor -- start 2023-06-23T00:35:42.810 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.806+0000 7f18d46e7700 1 -- start start 2023-06-23T00:35:42.812 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d46e7700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc156120 0x7f18cc1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.812 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d46e7700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f18cc0b1cc0 0x7f18cc0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.812 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d46e7700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18cc15afb0 con 0x7f18cc0c9f50 2023-06-23T00:35:42.812 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d46e7700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18cc06f470 con 0x7f18cc0b1cc0 2023-06-23T00:35:42.813 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d46e7700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0b1050 con 0x7f18cc156120 2023-06-23T00:35:42.813 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d2483700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc156120 0x7f18cc1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.813 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d2483700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc156120 0x7f18cc1564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3301/0 says I am v2:172.21.15.141:50360/0 (socket says 172.21.15.141:50360) 2023-06-23T00:35:42.813 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d2483700 1 -- 172.21.15.141:0/2647148468 learned_addr learned my addr 172.21.15.141:0/2647148468 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:42.813 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d2c84700 1 --2- 172.21.15.141:0/2647148468 >> v2:172.21.15.55:3300/0 conn(0x7f18cc0b1cc0 0x7f18cc0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.813 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d2483700 1 -- 172.21.15.141:0/2647148468 >> v2:172.21.15.55:3300/0 conn(0x7f18cc0b1cc0 msgr2=0x7f18cc0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:42.813 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d2483700 1 --2- 172.21.15.141:0/2647148468 >> v2:172.21.15.55:3300/0 conn(0x7f18cc0b1cc0 0x7f18cc0b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.813 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d2483700 1 -- 172.21.15.141:0/2647148468 >> v1:172.21.15.55:6789/0 conn(0x7f18cc0c9f50 legacy=0x7f18cc0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.814 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.808+0000 7f18d2483700 1 -- 172.21.15.141:0/2647148468 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc15a3a0 con 0x7f18cc156120 2023-06-23T00:35:42.814 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d2483700 1 --2- 172.21.15.141:0/2647148468 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc156120 0x7f18cc1564f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00bc50 tx=0x7f18bc00b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d81e6292088c6d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:42.814 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d1481700 1 -- 172.21.15.141:0/2647148468 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00dbf0 con 0x7f18cc156120 2023-06-23T00:35:42.814 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d1481700 1 -- 172.21.15.141:0/2647148468 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc00dd50 con 0x7f18cc156120 2023-06-23T00:35:42.814 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d1481700 1 -- 172.21.15.141:0/2647148468 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0196b0 con 0x7f18cc156120 2023-06-23T00:35:42.817 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d46e7700 1 -- 172.21.15.141:0/2647148468 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc156120 msgr2=0x7f18cc1564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.817 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d46e7700 1 --2- 172.21.15.141:0/2647148468 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc156120 0x7f18cc1564f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00bc50 tx=0x7f18bc00b4f0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.818 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d46e7700 1 -- 172.21.15.141:0/2647148468 shutdown_connections 2023-06-23T00:35:42.818 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d46e7700 1 --2- 172.21.15.141:0/2647148468 >> v2:172.21.15.55:3300/0 conn(0x7f18cc0b1cc0 0x7f18cc0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.818 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d46e7700 1 --2- 172.21.15.141:0/2647148468 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc156120 0x7f18cc1564f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.819 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d46e7700 1 -- 172.21.15.141:0/2647148468 >> 172.21.15.141:0/2647148468 conn(0x7f18cc0cde50 msgr2=0x7f18cc0aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:42.819 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d46e7700 1 -- 172.21.15.141:0/2647148468 shutdown_connections 2023-06-23T00:35:42.819 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.809+0000 7f18d46e7700 1 -- 172.21.15.141:0/2647148468 wait complete. 2023-06-23T00:35:42.819 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.810+0000 7f18d46e7700 1 Processor -- start 2023-06-23T00:35:42.820 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d46e7700 1 -- start start 2023-06-23T00:35:42.820 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d46e7700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f18cc0b1cc0 0x7f18cc152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.821 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d46e7700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc0c9f50 0x7f18cc0a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.821 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d46e7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18cc156120 0x7f18cc0add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:42.821 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d46e7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0acc70 con 0x7f18cc156120 2023-06-23T00:35:42.821 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d46e7700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0acf50 con 0x7f18cc0b1cc0 2023-06-23T00:35:42.821 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d46e7700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0ad230 con 0x7f18cc0c9f50 2023-06-23T00:35:42.821 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d1c82700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc0c9f50 0x7f18cc0a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.822 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d2483700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f18cc0b1cc0 0x7f18cc152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.822 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d1c82700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc0c9f50 0x7f18cc0a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3301/0 says I am v2:172.21.15.141:50364/0 (socket says 172.21.15.141:50364) 2023-06-23T00:35:42.822 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d1c82700 1 -- 172.21.15.141:0/1561540106 learned_addr learned my addr 172.21.15.141:0/1561540106 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:42.822 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.811+0000 7f18d2c84700 1 --2- 172.21.15.141:0/1561540106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18cc156120 0x7f18cc0add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:42.822 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18d1c82700 1 -- 172.21.15.141:0/1561540106 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f18cc0b1cc0 msgr2=0x7f18cc152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.822 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18d1c82700 1 --2- 172.21.15.141:0/1561540106 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f18cc0b1cc0 0x7f18cc152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.822 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18d1c82700 1 -- 172.21.15.141:0/1561540106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18cc156120 msgr2=0x7f18cc0add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.823 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18d1c82700 1 --2- 172.21.15.141:0/1561540106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18cc156120 0x7f18cc0add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.823 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18d1c82700 1 -- 172.21.15.141:0/1561540106 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc00b790 con 0x7f18cc0c9f50 2023-06-23T00:35:42.823 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18d2483700 1 --2- 172.21.15.141:0/1561540106 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f18cc0b1cc0 0x7f18cc152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:42.823 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18d2c84700 1 --2- 172.21.15.141:0/1561540106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18cc156120 0x7f18cc0add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:42.824 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18d1c82700 1 --2- 172.21.15.141:0/1561540106 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc0c9f50 0x7f18cc0a5de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f18c800c590 tx=0x7f18c800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:42.824 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18c27fc700 1 -- 172.21.15.141:0/1561540106 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8011590 con 0x7f18cc0c9f50 2023-06-23T00:35:42.824 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18c27fc700 1 -- 172.21.15.141:0/1561540106 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8011710 con 0x7f18cc0c9f50 2023-06-23T00:35:42.824 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18d46e7700 1 -- 172.21.15.141:0/1561540106 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18cc0ad510 con 0x7f18cc0c9f50 2023-06-23T00:35:42.825 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18c27fc700 1 -- 172.21.15.141:0/1561540106 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8011ad0 con 0x7f18cc0c9f50 2023-06-23T00:35:42.825 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18d46e7700 1 -- 172.21.15.141:0/1561540106 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18cc153470 con 0x7f18cc0c9f50 2023-06-23T00:35:42.825 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18c27fc700 1 -- 172.21.15.141:0/1561540106 <== mon.2 v2:172.21.15.141:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f18c801c6c0 con 0x7f18cc0c9f50 2023-06-23T00:35:42.826 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.812+0000 7f18c27fc700 1 -- 172.21.15.141:0/1561540106 <== mon.2 v2:172.21.15.141:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f18c80129a0 con 0x7f18cc0c9f50 2023-06-23T00:35:42.826 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.813+0000 7f18d46e7700 1 -- 172.21.15.141:0/1561540106 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18cc04c1b0 con 0x7f18cc0c9f50 2023-06-23T00:35:42.827 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.815+0000 7f18c27fc700 1 -- 172.21.15.141:0/1561540106 <== mon.2 v2:172.21.15.141:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f18c8017070 con 0x7f18cc0c9f50 2023-06-23T00:35:42.827 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:42.823+0000 7fde1d809180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T00:35:42.828 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:42.825+0000 7fa57af57180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T00:35:42.901 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.898+0000 7f18d46e7700 1 -- 172.21.15.141:0/1561540106 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "osd new", "uuid": "56cfcc58-3f4d-4637-ae5d-a2a52dbdc37f", "id": 5} v 0) v1 -- 0x7f18cc0ada30 con 0x7f18cc0c9f50 2023-06-23T00:35:42.907 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.903+0000 7f18c27fc700 1 -- 172.21.15.141:0/1561540106 <== mon.2 v2:172.21.15.141:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "56cfcc58-3f4d-4637-ae5d-a2a52dbdc37f", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f18cc0ada30 con 0x7f18cc0c9f50 2023-06-23T00:35:42.907 INFO:teuthology.orchestra.run.smithi141.stdout:5 2023-06-23T00:35:42.908 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.904+0000 7f18d46e7700 1 -- 172.21.15.141:0/1561540106 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc0c9f50 msgr2=0x7f18cc0a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:42.908 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.904+0000 7f18d46e7700 1 --2- 172.21.15.141:0/1561540106 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc0c9f50 0x7f18cc0a5de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f18c800c590 tx=0x7f18c800c8a0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.908 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.904+0000 7f18d46e7700 1 -- 172.21.15.141:0/1561540106 shutdown_connections 2023-06-23T00:35:42.909 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.904+0000 7f18d46e7700 1 --2- 172.21.15.141:0/1561540106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18cc156120 0x7f18cc0add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.909 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.904+0000 7f18d46e7700 1 --2- 172.21.15.141:0/1561540106 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f18cc0b1cc0 0x7f18cc152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.909 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.904+0000 7f18d46e7700 1 --2- 172.21.15.141:0/1561540106 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f18cc0c9f50 0x7f18cc0a5de0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:42.909 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.904+0000 7f18d46e7700 1 -- 172.21.15.141:0/1561540106 >> 172.21.15.141:0/1561540106 conn(0x7f18cc0cde50 msgr2=0x7f18cc158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:42.909 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.905+0000 7f18d46e7700 1 -- 172.21.15.141:0/1561540106 shutdown_connections 2023-06-23T00:35:42.909 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:42.905+0000 7f18d46e7700 1 -- 172.21.15.141:0/1561540106 wait complete. 2023-06-23T00:35:42.919 DEBUG:teuthology.orchestra.run.smithi141:> sudo ceph --cluster ceph osd new 49f9b91b-16c2-4394-928e-97446f5feea7 6 2023-06-23T00:35:42.968 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:42.964+0000 7fde1d809180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T00:35:42.971 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:42.969+0000 7fa57af57180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T00:35:43.035 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.031+0000 7f3bb06a1700 1 Processor -- start 2023-06-23T00:35:43.035 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.031+0000 7f3bb06a1700 1 -- start start 2023-06-23T00:35:43.041 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.032+0000 7f3bb06a1700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba80a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.042 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.032+0000 7f3bb06a1700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f3ba80ccb00 0x7f3ba80af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.042 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.032+0000 7f3bb06a1700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba80ae370 con 0x7f3ba80cddc0 2023-06-23T00:35:43.042 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.032+0000 7f3bb06a1700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80ce360 con 0x7f3ba80ccb00 2023-06-23T00:35:43.042 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.032+0000 7f3bb06a1700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80a9310 con 0x7f3ba80a8a00 2023-06-23T00:35:43.043 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.032+0000 7f3baec3e700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f3ba80ccb00 0x7f3ba80af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.043 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.032+0000 7f3bae43d700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba80a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.043 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.032+0000 7f3badc3c700 1 --1- >> v1:172.21.15.141:6789/0 conn(0x7f3ba80cddc0 0x7f3ba80cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.141:6789/0 says I am v1:172.21.15.141:59454/0 (socket says 172.21.15.141:59454) 2023-06-23T00:35:43.043 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.032+0000 7f3badc3c700 1 -- 172.21.15.141:0/523097755 learned_addr learned my addr 172.21.15.141:0/523097755 (peer_addr_for_me v1:172.21.15.141:0/0) 2023-06-23T00:35:43.044 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.033+0000 7f3baec3e700 1 -- 172.21.15.141:0/523097755 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 msgr2=0x7f3ba80a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:43.044 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.033+0000 7f3baec3e700 1 --2- 172.21.15.141:0/523097755 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba80a8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.044 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.033+0000 7f3baec3e700 1 -- 172.21.15.141:0/523097755 >> v1:172.21.15.141:6789/0 conn(0x7f3ba80cddc0 legacy=0x7f3ba80cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.044 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.033+0000 7f3baec3e700 1 -- 172.21.15.141:0/523097755 --> v2:172.21.15.141:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80ae610 con 0x7f3ba80ccb00 2023-06-23T00:35:43.044 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.033+0000 7f3bad43b700 1 -- 172.21.15.141:0/523097755 <== mon.1 v1:172.21.15.141:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216761157 0 0) 0x7f3ba80ae370 con 0x7f3ba80cddc0 2023-06-23T00:35:43.044 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.033+0000 7f3bae43d700 1 --2- 172.21.15.141:0/523097755 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba80a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:43.045 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.033+0000 7f3baec3e700 1 --2- 172.21.15.141:0/523097755 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3ba80ccb00 0x7f3ba80af3d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009e60 tx=0x7f3ba0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef3210fe432eaae9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:43.045 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.033+0000 7f3bad43b700 1 -- 172.21.15.141:0/523097755 <== mon.1 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000e070 con 0x7f3ba80ccb00 2023-06-23T00:35:43.045 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.033+0000 7f3bad43b700 1 -- 172.21.15.141:0/523097755 <== mon.1 v2:172.21.15.141:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba000bbf0 con 0x7f3ba80ccb00 2023-06-23T00:35:43.045 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.033+0000 7f3bad43b700 1 -- 172.21.15.141:0/523097755 <== mon.1 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0012430 con 0x7f3ba80ccb00 2023-06-23T00:35:43.045 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.034+0000 7f3bb06a1700 1 -- 172.21.15.141:0/523097755 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3ba80ccb00 msgr2=0x7f3ba80af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.045 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.034+0000 7f3bb06a1700 1 --2- 172.21.15.141:0/523097755 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3ba80ccb00 0x7f3ba80af3d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009e60 tx=0x7f3ba0009780 comp rx=0 tx=0).stop 2023-06-23T00:35:43.046 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.034+0000 7f3bb06a1700 1 -- 172.21.15.141:0/523097755 shutdown_connections 2023-06-23T00:35:43.046 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.034+0000 7f3bb06a1700 1 --2- 172.21.15.141:0/523097755 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3ba80ccb00 0x7f3ba80af3d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.046 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.034+0000 7f3bb06a1700 1 --2- 172.21.15.141:0/523097755 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba80a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.046 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.034+0000 7f3bb06a1700 1 -- 172.21.15.141:0/523097755 >> 172.21.15.141:0/523097755 conn(0x7f3ba8067bf0 msgr2=0x7f3ba80ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:43.046 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.034+0000 7f3bb06a1700 1 -- 172.21.15.141:0/523097755 shutdown_connections 2023-06-23T00:35:43.046 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.034+0000 7f3bb06a1700 1 -- 172.21.15.141:0/523097755 wait complete. 2023-06-23T00:35:43.047 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.035+0000 7f3bb06a1700 1 Processor -- start 2023-06-23T00:35:43.047 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.035+0000 7f3bb06a1700 1 -- start start 2023-06-23T00:35:43.047 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bb06a1700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba809b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.048 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bb06a1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ba80ccb00 0x7f3ba814c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.048 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bb06a1700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3ba80cddc0 0x7f3ba815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.048 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bb06a1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba809a0b0 con 0x7f3ba80ccb00 2023-06-23T00:35:43.048 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bb06a1700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80994a0 con 0x7f3ba80cddc0 2023-06-23T00:35:43.048 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bb06a1700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8099780 con 0x7f3ba80a8a00 2023-06-23T00:35:43.049 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bae43d700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba809b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.049 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3baec3e700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3ba80cddc0 0x7f3ba815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.049 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bae43d700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba809b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3301/0 says I am v2:172.21.15.141:50388/0 (socket says 172.21.15.141:50388) 2023-06-23T00:35:43.049 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bae43d700 1 -- 172.21.15.141:0/70765162 learned_addr learned my addr 172.21.15.141:0/70765162 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:43.049 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3badc3c700 1 --2- 172.21.15.141:0/70765162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ba80ccb00 0x7f3ba814c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.050 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3baec3e700 1 -- 172.21.15.141:0/70765162 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 msgr2=0x7f3ba809b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.050 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3baec3e700 1 --2- 172.21.15.141:0/70765162 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba809b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.050 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3baec3e700 1 -- 172.21.15.141:0/70765162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ba80ccb00 msgr2=0x7f3ba814c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.050 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3baec3e700 1 --2- 172.21.15.141:0/70765162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ba80ccb00 0x7f3ba814c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.051 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3baec3e700 1 -- 172.21.15.141:0/70765162 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b980093c0 con 0x7f3ba80cddc0 2023-06-23T00:35:43.051 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bae43d700 1 --2- 172.21.15.141:0/70765162 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba809b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:43.051 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3baec3e700 1 --2- 172.21.15.141:0/70765162 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3ba80cddc0 0x7f3ba815bcf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0016000 tx=0x7f3ba0019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:43.051 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3b9e7fc700 1 -- 172.21.15.141:0/70765162 <== mon.1 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0012e10 con 0x7f3ba80cddc0 2023-06-23T00:35:43.051 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3b9e7fc700 1 -- 172.21.15.141:0/70765162 <== mon.1 v2:172.21.15.141:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba001d630 con 0x7f3ba80cddc0 2023-06-23T00:35:43.051 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3b9e7fc700 1 -- 172.21.15.141:0/70765162 <== mon.1 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba001d9d0 con 0x7f3ba80cddc0 2023-06-23T00:35:43.052 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bb06a1700 1 -- 172.21.15.141:0/70765162 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba0009a20 con 0x7f3ba80cddc0 2023-06-23T00:35:43.052 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3bb06a1700 1 -- 172.21.15.141:0/70765162 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba8099d40 con 0x7f3ba80cddc0 2023-06-23T00:35:43.052 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.036+0000 7f3b9e7fc700 1 -- 172.21.15.141:0/70765162 <== mon.1 v2:172.21.15.141:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3ba0024070 con 0x7f3ba80cddc0 2023-06-23T00:35:43.052 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.037+0000 7f3b9e7fc700 1 -- 172.21.15.141:0/70765162 <== mon.1 v2:172.21.15.141:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f3ba0027cf0 con 0x7f3ba80cddc0 2023-06-23T00:35:43.053 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.037+0000 7f3bb06a1700 1 -- 172.21.15.141:0/70765162 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba804c1b0 con 0x7f3ba80cddc0 2023-06-23T00:35:43.053 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.041+0000 7f3b9e7fc700 1 -- 172.21.15.141:0/70765162 <== mon.1 v2:172.21.15.141:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f3ba00476e0 con 0x7f3ba80cddc0 2023-06-23T00:35:43.068 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:43.064+0000 7fde1d809180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T00:35:43.074 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:43.070+0000 7fa57af57180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T00:35:43.126 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.123+0000 7f3bb06a1700 1 -- 172.21.15.141:0/70765162 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_command({"prefix": "osd new", "uuid": "49f9b91b-16c2-4394-928e-97446f5feea7", "id": 6} v 0) v1 -- 0x7f3ba806a510 con 0x7f3ba80cddc0 2023-06-23T00:35:43.133 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.131+0000 7f3b9e7fc700 1 -- 172.21.15.141:0/70765162 <== mon.1 v2:172.21.15.141:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "49f9b91b-16c2-4394-928e-97446f5feea7", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3ba0036030 con 0x7f3ba80cddc0 2023-06-23T00:35:43.134 INFO:teuthology.orchestra.run.smithi141.stdout:6 2023-06-23T00:35:43.135 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.132+0000 7f3bb06a1700 1 -- 172.21.15.141:0/70765162 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3ba80cddc0 msgr2=0x7f3ba815bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.135 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.132+0000 7f3bb06a1700 1 --2- 172.21.15.141:0/70765162 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3ba80cddc0 0x7f3ba815bcf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0016000 tx=0x7f3ba0019000 comp rx=0 tx=0).stop 2023-06-23T00:35:43.135 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.132+0000 7f3bb06a1700 1 -- 172.21.15.141:0/70765162 shutdown_connections 2023-06-23T00:35:43.135 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.132+0000 7f3bb06a1700 1 --2- 172.21.15.141:0/70765162 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f3ba80cddc0 0x7f3ba815bcf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.135 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.132+0000 7f3bb06a1700 1 --2- 172.21.15.141:0/70765162 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f3ba80a8a00 0x7f3ba809b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.136 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.132+0000 7f3bb06a1700 1 --2- 172.21.15.141:0/70765162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ba80ccb00 0x7f3ba814c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.136 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.132+0000 7f3bb06a1700 1 -- 172.21.15.141:0/70765162 >> 172.21.15.141:0/70765162 conn(0x7f3ba8067bf0 msgr2=0x7f3ba8068520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:43.136 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.132+0000 7f3bb06a1700 1 -- 172.21.15.141:0/70765162 shutdown_connections 2023-06-23T00:35:43.136 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.132+0000 7f3bb06a1700 1 -- 172.21.15.141:0/70765162 wait complete. 2023-06-23T00:35:43.146 DEBUG:teuthology.orchestra.run.smithi141:> sudo ceph --cluster ceph osd new e9806e67-d1e0-49c5-a3e4-898bd4f840e7 7 2023-06-23T00:35:43.262 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.259+0000 7efdc3fff700 1 Processor -- start 2023-06-23T00:35:43.263 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.259+0000 7efdc3fff700 1 -- start start 2023-06-23T00:35:43.264 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.260+0000 7efdc3fff700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc4157870 0x7efdc40ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.264 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.260+0000 7efdc3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efdc40cd420 0x7efdc40c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.264 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.260+0000 7efdc3fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc40b17d0 con 0x7efdc41574a0 2023-06-23T00:35:43.264 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.260+0000 7efdc3fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc4157cc0 con 0x7efdc40cd420 2023-06-23T00:35:43.265 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.260+0000 7efdc3fff700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc40c3460 con 0x7efdc4157870 2023-06-23T00:35:43.265 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.260+0000 7efdba7fc700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc4157870 0x7efdc40ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.265 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.260+0000 7efdba7fc700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc4157870 0x7efdc40ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3301/0 says I am v2:172.21.15.141:50402/0 (socket says 172.21.15.141:50402) 2023-06-23T00:35:43.265 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.260+0000 7efdba7fc700 1 -- 172.21.15.141:0/1420717781 learned_addr learned my addr 172.21.15.141:0/1420717781 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:43.265 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc37fe700 1 --2- 172.21.15.141:0/1420717781 >> v2:172.21.15.55:3300/0 conn(0x7efdc40cd420 0x7efdc40c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.265 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdba7fc700 1 -- 172.21.15.141:0/1420717781 >> v2:172.21.15.55:3300/0 conn(0x7efdc40cd420 msgr2=0x7efdc40c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:43.265 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdba7fc700 1 --2- 172.21.15.141:0/1420717781 >> v2:172.21.15.55:3300/0 conn(0x7efdc40cd420 0x7efdc40c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.265 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdba7fc700 1 -- 172.21.15.141:0/1420717781 >> v1:172.21.15.55:6789/0 conn(0x7efdc41574a0 legacy=0x7efdc40b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:43.266 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdba7fc700 1 -- 172.21.15.141:0/1420717781 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc40c3740 con 0x7efdc4157870 2023-06-23T00:35:43.266 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdba7fc700 1 --2- 172.21.15.141:0/1420717781 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc4157870 0x7efdc40ccee0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efdac0099b0 tx=0x7efdac009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ba60e70d7665a74 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:43.266 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc27fc700 1 -- 172.21.15.141:0/1420717781 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac00b820 con 0x7efdc4157870 2023-06-23T00:35:43.266 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc27fc700 1 -- 172.21.15.141:0/1420717781 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac00b980 con 0x7efdc4157870 2023-06-23T00:35:43.266 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc27fc700 1 -- 172.21.15.141:0/1420717781 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac00bd20 con 0x7efdc4157870 2023-06-23T00:35:43.268 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc3fff700 1 -- 172.21.15.141:0/1420717781 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc4157870 msgr2=0x7efdc40ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.269 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc3fff700 1 --2- 172.21.15.141:0/1420717781 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc4157870 0x7efdc40ccee0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efdac0099b0 tx=0x7efdac009250 comp rx=0 tx=0).stop 2023-06-23T00:35:43.269 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc3fff700 1 -- 172.21.15.141:0/1420717781 shutdown_connections 2023-06-23T00:35:43.269 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc3fff700 1 --2- 172.21.15.141:0/1420717781 >> v2:172.21.15.55:3300/0 conn(0x7efdc40cd420 0x7efdc40c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.269 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc3fff700 1 --2- 172.21.15.141:0/1420717781 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc4157870 0x7efdc40ccee0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.269 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc3fff700 1 -- 172.21.15.141:0/1420717781 >> 172.21.15.141:0/1420717781 conn(0x7efdc4087f50 msgr2=0x7efdc40ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:43.269 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc3fff700 1 -- 172.21.15.141:0/1420717781 shutdown_connections 2023-06-23T00:35:43.269 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.261+0000 7efdc3fff700 1 -- 172.21.15.141:0/1420717781 wait complete. 2023-06-23T00:35:43.269 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.263+0000 7efdc3fff700 1 Processor -- start 2023-06-23T00:35:43.270 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.263+0000 7efdc3fff700 1 -- start start 2023-06-23T00:35:43.270 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.263+0000 7efdc3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc40cd420 0x7efdc40c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.270 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.263+0000 7efdc3fff700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc41574a0 0x7efdc41511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.270 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.263+0000 7efdc3fff700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdc4157870 0x7efdc4167d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.270 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.263+0000 7efdc3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc40c58a0 con 0x7efdc40cd420 2023-06-23T00:35:43.270 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.263+0000 7efdc3fff700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc40c6270 con 0x7efdc4157870 2023-06-23T00:35:43.270 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.263+0000 7efdc3fff700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc4168800 con 0x7efdc41574a0 2023-06-23T00:35:43.271 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdc4157870 0x7efdc4167d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.271 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdba7fc700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc41574a0 0x7efdc41511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.271 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdba7fc700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc41574a0 0x7efdc41511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3301/0 says I am v2:172.21.15.141:50410/0 (socket says 172.21.15.141:50410) 2023-06-23T00:35:43.271 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdc4157870 0x7efdc4167d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.141:3300/0 says I am v2:172.21.15.141:45792/0 (socket says 172.21.15.141:45792) 2023-06-23T00:35:43.271 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdba7fc700 1 -- 172.21.15.141:0/4091197593 learned_addr learned my addr 172.21.15.141:0/4091197593 (peer_addr_for_me v2:172.21.15.141:0/0) 2023-06-23T00:35:43.271 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdc2ffd700 1 --2- 172.21.15.141:0/4091197593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc40cd420 0x7efdc40c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.271 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdba7fc700 1 -- 172.21.15.141:0/4091197593 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdc4157870 msgr2=0x7efdc4167d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.271 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdba7fc700 1 --2- 172.21.15.141:0/4091197593 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdc4157870 0x7efdc4167d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.272 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdba7fc700 1 -- 172.21.15.141:0/4091197593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc40cd420 msgr2=0x7efdc40c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.272 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdba7fc700 1 --2- 172.21.15.141:0/4091197593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc40cd420 0x7efdc40c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.272 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdba7fc700 1 -- 172.21.15.141:0/4091197593 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac0094f0 con 0x7efdc41574a0 2023-06-23T00:35:43.272 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdc37fe700 1 --2- 172.21.15.141:0/4091197593 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdc4157870 0x7efdc4167d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:43.272 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdba7fc700 1 --2- 172.21.15.141:0/4091197593 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc41574a0 0x7efdc41511e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efdac009290 tx=0x7efdac016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:43.272 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdbb7fe700 1 -- 172.21.15.141:0/4091197593 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac01b470 con 0x7efdc41574a0 2023-06-23T00:35:43.272 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdbb7fe700 1 -- 172.21.15.141:0/4091197593 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac01b5f0 con 0x7efdc41574a0 2023-06-23T00:35:43.273 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdc3fff700 1 -- 172.21.15.141:0/4091197593 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc4168ae0 con 0x7efdc41574a0 2023-06-23T00:35:43.273 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdbb7fe700 1 -- 172.21.15.141:0/4091197593 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac01b9b0 con 0x7efdc41574a0 2023-06-23T00:35:43.273 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdc3fff700 1 -- 172.21.15.141:0/4091197593 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc4151720 con 0x7efdc41574a0 2023-06-23T00:35:43.273 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdbb7fe700 1 -- 172.21.15.141:0/4091197593 <== mon.2 v2:172.21.15.141:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7efdac01c6d0 con 0x7efdc41574a0 2023-06-23T00:35:43.273 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.264+0000 7efdbb7fe700 1 -- 172.21.15.141:0/4091197593 <== mon.2 v2:172.21.15.141:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7efdac01bda0 con 0x7efdc41574a0 2023-06-23T00:35:43.273 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.265+0000 7efdc3fff700 1 -- 172.21.15.141:0/4091197593 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc40c5a10 con 0x7efdc41574a0 2023-06-23T00:35:43.274 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.267+0000 7efdbb7fe700 1 -- 172.21.15.141:0/4091197593 <== mon.2 v2:172.21.15.141:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7efdac02a060 con 0x7efdc41574a0 2023-06-23T00:35:43.352 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.349+0000 7efdc3fff700 1 -- 172.21.15.141:0/4091197593 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e9806e67-d1e0-49c5-a3e4-898bd4f840e7", "id": 7} v 0) v1 -- 0x7efdc40c5390 con 0x7efdc41574a0 2023-06-23T00:35:43.363 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.355+0000 7efdbb7fe700 1 -- 172.21.15.141:0/4091197593 <== mon.2 v2:172.21.15.141:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e9806e67-d1e0-49c5-a3e4-898bd4f840e7", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7efdc40c5390 con 0x7efdc41574a0 2023-06-23T00:35:43.363 INFO:teuthology.orchestra.run.smithi141.stdout:7 2023-06-23T00:35:43.365 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.356+0000 7efdc3fff700 1 -- 172.21.15.141:0/4091197593 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc41574a0 msgr2=0x7efdc41511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.365 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.356+0000 7efdc3fff700 1 --2- 172.21.15.141:0/4091197593 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc41574a0 0x7efdc41511e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efdac009290 tx=0x7efdac016000 comp rx=0 tx=0).stop 2023-06-23T00:35:43.366 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.356+0000 7efdc3fff700 1 -- 172.21.15.141:0/4091197593 shutdown_connections 2023-06-23T00:35:43.366 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.356+0000 7efdc3fff700 1 --2- 172.21.15.141:0/4091197593 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efdc4157870 0x7efdc4167d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.366 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.356+0000 7efdc3fff700 1 --2- 172.21.15.141:0/4091197593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc40cd420 0x7efdc40c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.366 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.356+0000 7efdc3fff700 1 --2- 172.21.15.141:0/4091197593 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efdc41574a0 0x7efdc41511e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.367 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.356+0000 7efdc3fff700 1 -- 172.21.15.141:0/4091197593 >> 172.21.15.141:0/4091197593 conn(0x7efdc4087f50 msgr2=0x7efdc4156690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:43.367 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.356+0000 7efdc3fff700 1 -- 172.21.15.141:0/4091197593 shutdown_connections 2023-06-23T00:35:43.367 INFO:teuthology.orchestra.run.smithi141.stderr:2023-06-23T00:35:43.356+0000 7efdc3fff700 1 -- 172.21.15.141:0/4091197593 wait complete. 2023-06-23T00:35:43.371 INFO:tasks.ceph.osd.0:Restarting daemon 2023-06-23T00:35:43.372 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-06-23T00:35:43.374 INFO:tasks.ceph.osd.0:Started 2023-06-23T00:35:43.375 INFO:tasks.ceph.osd.1:Restarting daemon 2023-06-23T00:35:43.376 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-06-23T00:35:43.378 INFO:tasks.ceph.osd.1:Started 2023-06-23T00:35:43.378 INFO:tasks.ceph.osd.2:Restarting daemon 2023-06-23T00:35:43.379 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-06-23T00:35:43.381 INFO:tasks.ceph.osd.2:Started 2023-06-23T00:35:43.382 INFO:tasks.ceph.osd.3:Restarting daemon 2023-06-23T00:35:43.382 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-06-23T00:35:43.386 INFO:tasks.ceph.osd.3:Started 2023-06-23T00:35:43.386 INFO:tasks.ceph.osd.4:Restarting daemon 2023-06-23T00:35:43.387 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-06-23T00:35:43.389 INFO:tasks.ceph.osd.4:Started 2023-06-23T00:35:43.390 INFO:tasks.ceph.osd.5:Restarting daemon 2023-06-23T00:35:43.390 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-06-23T00:35:43.392 INFO:tasks.ceph.osd.5:Started 2023-06-23T00:35:43.393 INFO:tasks.ceph.osd.6:Restarting daemon 2023-06-23T00:35:43.393 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-06-23T00:35:43.395 INFO:tasks.ceph.osd.6:Started 2023-06-23T00:35:43.395 INFO:tasks.ceph.osd.7:Restarting daemon 2023-06-23T00:35:43.396 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-06-23T00:35:43.398 INFO:tasks.ceph.osd.7:Started 2023-06-23T00:35:43.399 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T00:35:43.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.545+0000 7f12d17c6700 1 Processor -- start 2023-06-23T00:35:43.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.545+0000 7f12d17c6700 1 -- start start 2023-06-23T00:35:43.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.545+0000 7f12d17c6700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f12cc0ad470 0x7f12cc0abf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.545+0000 7f12d17c6700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc15a280 con 0x7f12cc0af570 2023-06-23T00:35:43.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.545+0000 7f12d17c6700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0afa00 con 0x7f12cc1508d0 2023-06-23T00:35:43.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.545+0000 7f12d17c6700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc0afce0 con 0x7f12cc0ad470 2023-06-23T00:35:43.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.546+0000 7f12ca7fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f12cc0af570 0x7f12cc0acd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55430/0 (socket says 172.21.15.55:55430) 2023-06-23T00:35:43.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.546+0000 7f12ca7fc700 1 -- 172.21.15.55:0/3818049243 learned_addr learned my addr 172.21.15.55:0/3818049243 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:43.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.546+0000 7f12cb7fe700 1 --2- 172.21.15.55:0/3818049243 >> v2:172.21.15.141:3300/0 conn(0x7f12cc0ad470 0x7f12cc0abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.546+0000 7f12c9ffb700 1 -- 172.21.15.55:0/3818049243 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216943915 0 0) 0x7f12cc15a280 con 0x7f12cc0af570 2023-06-23T00:35:43.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.546+0000 7f12c9ffb700 1 -- 172.21.15.55:0/3818049243 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b4003570 con 0x7f12cc0af570 2023-06-23T00:35:43.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.546+0000 7f12cb7fe700 1 -- 172.21.15.55:0/3818049243 >> v1:172.21.15.141:6789/0 conn(0x7f12cc1508d0 legacy=0x7f12cc0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.546+0000 7f12c9ffb700 1 -- 172.21.15.55:0/3818049243 <== mon.1 v1:172.21.15.141:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001194246 0 0) 0x7f12cc0afa00 con 0x7f12cc1508d0 2023-06-23T00:35:43.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.546+0000 7f12cb7fe700 1 -- 172.21.15.55:0/3818049243 >> v1:172.21.15.55:6789/0 conn(0x7f12cc0af570 legacy=0x7f12cc0acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.546+0000 7f12cb7fe700 1 -- 172.21.15.55:0/3818049243 --> v2:172.21.15.141:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0affc0 con 0x7f12cc0ad470 2023-06-23T00:35:43.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.546+0000 7f12cb7fe700 1 --2- 172.21.15.55:0/3818049243 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12cc0ad470 0x7f12cc0abf00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009e60 tx=0x7f12c4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97efd1112350caf4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:43.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.547+0000 7f12c9ffb700 1 -- 172.21.15.55:0/3818049243 <== mon.1 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400e070 con 0x7f12cc0ad470 2023-06-23T00:35:43.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.547+0000 7f12c9ffb700 1 -- 172.21.15.55:0/3818049243 <== mon.1 v2:172.21.15.141:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c400bb60 con 0x7f12cc0ad470 2023-06-23T00:35:43.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.547+0000 7f12c9ffb700 1 -- 172.21.15.55:0/3818049243 <== mon.1 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4012430 con 0x7f12cc0ad470 2023-06-23T00:35:43.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.547+0000 7f12d17c6700 1 -- 172.21.15.55:0/3818049243 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12cc0ad470 msgr2=0x7f12cc0abf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.547+0000 7f12d17c6700 1 --2- 172.21.15.55:0/3818049243 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12cc0ad470 0x7f12cc0abf00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009e60 tx=0x7f12c4009780 comp rx=0 tx=0).stop 2023-06-23T00:35:43.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.547+0000 7f12d17c6700 1 -- 172.21.15.55:0/3818049243 shutdown_connections 2023-06-23T00:35:43.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.547+0000 7f12d17c6700 1 --2- 172.21.15.55:0/3818049243 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12cc0ad470 0x7f12cc0abf00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.547+0000 7f12d17c6700 1 -- 172.21.15.55:0/3818049243 >> 172.21.15.55:0/3818049243 conn(0x7f12cc067bf0 msgr2=0x7f12cc087f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:43.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.547+0000 7f12d17c6700 1 -- 172.21.15.55:0/3818049243 shutdown_connections 2023-06-23T00:35:43.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.547+0000 7f12d17c6700 1 -- 172.21.15.55:0/3818049243 wait complete. 2023-06-23T00:35:43.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.548+0000 7f12d17c6700 1 Processor -- start 2023-06-23T00:35:43.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.548+0000 7f12d17c6700 1 -- start start 2023-06-23T00:35:43.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12d17c6700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f12cc0ad470 0x7f12cc169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12d17c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12cc0af570 0x7f12cc16c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12d17c6700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12cc1508d0 0x7f12cc1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:43.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12d17c6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc155ea0 con 0x7f12cc0af570 2023-06-23T00:35:43.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12d17c6700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc152a80 con 0x7f12cc1508d0 2023-06-23T00:35:43.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12d17c6700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1550d0 con 0x7f12cc0ad470 2023-06-23T00:35:43.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12ca7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12cc0af570 0x7f12cc16c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12ca7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12cc0af570 0x7f12cc16c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50782/0 (socket says 172.21.15.55:50782) 2023-06-23T00:35:43.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12ca7fc700 1 -- 172.21.15.55:0/315039604 learned_addr learned my addr 172.21.15.55:0/315039604 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:43.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12cb7fe700 1 --2- 172.21.15.55:0/315039604 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12cc1508d0 0x7f12cc1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.549+0000 7f12caffd700 1 --2- 172.21.15.55:0/315039604 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f12cc0ad470 0x7f12cc169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:43.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12ca7fc700 1 -- 172.21.15.55:0/315039604 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f12cc0ad470 msgr2=0x7f12cc169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12ca7fc700 1 --2- 172.21.15.55:0/315039604 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f12cc0ad470 0x7f12cc169a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12ca7fc700 1 -- 172.21.15.55:0/315039604 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12cc1508d0 msgr2=0x7f12cc1520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12ca7fc700 1 --2- 172.21.15.55:0/315039604 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12cc1508d0 0x7f12cc1520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12ca7fc700 1 -- 172.21.15.55:0/315039604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4009a20 con 0x7f12cc0af570 2023-06-23T00:35:43.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12cb7fe700 1 --2- 172.21.15.55:0/315039604 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12cc1508d0 0x7f12cc1520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:43.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12ca7fc700 1 --2- 172.21.15.55:0/315039604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12cc0af570 0x7f12cc16c8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12c00011e0 tx=0x7f12c0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:43.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12b2ffd700 1 -- 172.21.15.55:0/315039604 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0005b30 con 0x7f12cc0af570 2023-06-23T00:35:43.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12b2ffd700 1 -- 172.21.15.55:0/315039604 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c0005c90 con 0x7f12cc0af570 2023-06-23T00:35:43.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12d17c6700 1 -- 172.21.15.55:0/315039604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc155320 con 0x7f12cc0af570 2023-06-23T00:35:43.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12b2ffd700 1 -- 172.21.15.55:0/315039604 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0007300 con 0x7f12cc0af570 2023-06-23T00:35:43.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12d17c6700 1 -- 172.21.15.55:0/315039604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc1558e0 con 0x7f12cc0af570 2023-06-23T00:35:43.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12b2ffd700 1 -- 172.21.15.55:0/315039604 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f12c001c440 con 0x7f12cc0af570 2023-06-23T00:35:43.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.550+0000 7f12b2ffd700 1 -- 172.21.15.55:0/315039604 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f12c0005e00 con 0x7f12cc0af570 2023-06-23T00:35:43.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.551+0000 7f12d17c6700 1 -- 172.21.15.55:0/315039604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc04c1b0 con 0x7f12cc0af570 2023-06-23T00:35:43.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.553+0000 7f12b2ffd700 1 -- 172.21.15.55:0/315039604 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f12c0017070 con 0x7f12cc0af570 2023-06-23T00:35:43.638 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:43.635+0000 7fa57af57180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T00:35:43.648 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:43.644+0000 7fde1d809180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T00:35:43.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.645+0000 7f12d17c6700 1 -- 172.21.15.55:0/315039604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12cc152bf0 con 0x7f12cc0af570 2023-06-23T00:35:43.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.646+0000 7f12b2ffd700 1 -- 172.21.15.55:0/315039604 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f12cc152bf0 con 0x7f12cc0af570 2023-06-23T00:35:43.650 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:43.650 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":10,"fsid":"c0a83a0b-dd71-486d-bdc4-a203f7ef404f","created":"2023-06-23T00:35:40.890983+0000","modified":"2023-06-23T00:35:43.351694+0000","last_up_change":"0.000000","last_in_change":"2023-06-23T00:35:43.351694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d4590eaf-cc24-4ff1-8f1b-9637138bd300","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"518b1bc8-fb1c-405f-8346-fda7978829a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"636a04f1-2379-43ca-9f02-0ee33d2af90a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b893aff1-9f4d-42ca-9af6-42fd6ba3d92c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"259d9777-c081-4c0b-9d83-46d348872e01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"56cfcc58-3f4d-4637-ae5d-a2a52dbdc37f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"49f9b91b-16c2-4394-928e-97446f5feea7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e9806e67-d1e0-49c5-a3e4-898bd4f840e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T00:35:43.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.647+0000 7f12d17c6700 1 -- 172.21.15.55:0/315039604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12cc0af570 msgr2=0x7f12cc16c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:43.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.647+0000 7f12d17c6700 1 --2- 172.21.15.55:0/315039604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12cc0af570 0x7f12cc16c8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12c00011e0 tx=0x7f12c0007df0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.647+0000 7f12d17c6700 1 -- 172.21.15.55:0/315039604 shutdown_connections 2023-06-23T00:35:43.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.647+0000 7f12d17c6700 1 --2- 172.21.15.55:0/315039604 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f12cc1508d0 0x7f12cc1520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.647+0000 7f12d17c6700 1 --2- 172.21.15.55:0/315039604 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f12cc0ad470 0x7f12cc169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.647+0000 7f12d17c6700 1 --2- 172.21.15.55:0/315039604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12cc0af570 0x7f12cc16c8d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:43.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.647+0000 7f12d17c6700 1 -- 172.21.15.55:0/315039604 >> 172.21.15.55:0/315039604 conn(0x7f12cc067bf0 msgr2=0x7f12cc06b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:43.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.647+0000 7f12d17c6700 1 -- 172.21.15.55:0/315039604 shutdown_connections 2023-06-23T00:35:43.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:43.647+0000 7f12d17c6700 1 -- 172.21.15.55:0/315039604 wait complete. 2023-06-23T00:35:43.663 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-06-23T00:35:43.663 INFO:tasks.ceph:Waiting for OSDs to come up 2023-06-23T00:35:43.715 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:43.712+0000 7fa57af57180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T00:35:43.732 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:43.727+0000 7fde1d809180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T00:35:43.794 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:43.791+0000 7fa57af57180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T00:35:43.812 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:43.808+0000 7fde1d809180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T00:35:43.985 INFO:tasks.ceph.osd.2.smithi055.stderr:2023-06-23T00:35:43.981+0000 7ffb6dd40540 -1 Falling back to public interface 2023-06-23T00:35:43.990 INFO:tasks.ceph.osd.5.smithi141.stderr:2023-06-23T00:35:43.987+0000 7f4c7002e540 -1 Falling back to public interface 2023-06-23T00:35:43.997 INFO:tasks.ceph.osd.1.smithi055.stderr:2023-06-23T00:35:43.993+0000 7f9d4e098540 -1 Falling back to public interface 2023-06-23T00:35:44.003 INFO:tasks.ceph.osd.7.smithi141.stderr:2023-06-23T00:35:43.999+0000 7f6fdf5c8540 -1 Falling back to public interface 2023-06-23T00:35:44.006 INFO:tasks.ceph.osd.0.smithi055.stderr:2023-06-23T00:35:44.002+0000 7f9634bfb540 -1 Falling back to public interface 2023-06-23T00:35:44.009 INFO:tasks.ceph.osd.4.smithi141.stderr:2023-06-23T00:35:44.006+0000 7f7d28a8f540 -1 Falling back to public interface 2023-06-23T00:35:44.010 INFO:tasks.ceph.osd.6.smithi141.stderr:2023-06-23T00:35:44.007+0000 7f9ab6348540 -1 Falling back to public interface 2023-06-23T00:35:44.017 INFO:tasks.ceph.osd.3.smithi055.stderr:2023-06-23T00:35:44.013+0000 7fb8ba8ae540 -1 Falling back to public interface 2023-06-23T00:35:44.388 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:44.385+0000 7fa57af57180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T00:35:44.423 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:44.420+0000 7fde1d809180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T00:35:44.468 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-23T00:35:44.526 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:44.522+0000 7fa57af57180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T00:35:44.569 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:44.565+0000 7fde1d809180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T00:35:44.578 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.574+0000 7f46140c6700 1 Processor -- start 2023-06-23T00:35:44.578 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.574+0000 7f46140c6700 1 -- start start 2023-06-23T00:35:44.580 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f46140c6700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f460c0b1e90 0x7f460c0b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:44.580 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f46140c6700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f460c157c00 0x7f460c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:44.580 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f46140c6700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f460c0cd160 0x7f460c0cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:44.580 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f46140c6700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c0cdb20 con 0x7f460c0cd160 2023-06-23T00:35:44.581 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f46140c6700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c158d80 con 0x7f460c157c00 2023-06-23T00:35:44.581 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f46140c6700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c158170 con 0x7f460c0b1e90 2023-06-23T00:35:44.581 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f4612663700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f460c0cd160 0x7f460c0cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:44.581 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f4612663700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f460c0cd160 0x7f460c0cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50786/0 (socket says 172.21.15.55:50786) 2023-06-23T00:35:44.581 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f4612663700 1 -- 172.21.15.55:0/2297016159 learned_addr learned my addr 172.21.15.55:0/2297016159 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:44.581 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f4611661700 1 --2- 172.21.15.55:0/2297016159 >> v2:172.21.15.141:3300/0 conn(0x7f460c157c00 0x7f460c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:44.581 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.575+0000 7f4611e62700 1 --2- 172.21.15.55:0/2297016159 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f460c0b1e90 0x7f460c0b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:44.582 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.576+0000 7f4612663700 1 -- 172.21.15.55:0/2297016159 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f460c0b1e90 msgr2=0x7f460c0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:44.582 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.576+0000 7f4612663700 1 --2- 172.21.15.55:0/2297016159 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f460c0b1e90 0x7f460c0b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:44.582 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.576+0000 7f4612663700 1 -- 172.21.15.55:0/2297016159 >> v2:172.21.15.141:3300/0 conn(0x7f460c157c00 msgr2=0x7f460c06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:44.582 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.576+0000 7f4612663700 1 --2- 172.21.15.55:0/2297016159 >> v2:172.21.15.141:3300/0 conn(0x7f460c157c00 0x7f460c06eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:44.582 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.576+0000 7f4612663700 1 -- 172.21.15.55:0/2297016159 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c158450 con 0x7f460c0cd160 2023-06-23T00:35:44.582 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.576+0000 7f4612663700 1 --2- 172.21.15.55:0/2297016159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c0cd160 0x7f460c0cd5e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f460c06ded0 tx=0x7f4604009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fa7a0d9adc8aa6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:44.582 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.576+0000 7f4610e60700 1 -- 172.21.15.55:0/2297016159 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400e070 con 0x7f460c0cd160 2023-06-23T00:35:44.583 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.576+0000 7f4610e60700 1 -- 172.21.15.55:0/2297016159 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460400b970 con 0x7f460c0cd160 2023-06-23T00:35:44.583 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.576+0000 7f4610e60700 1 -- 172.21.15.55:0/2297016159 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040092e0 con 0x7f460c0cd160 2023-06-23T00:35:44.585 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.577+0000 7f46140c6700 1 -- 172.21.15.55:0/2297016159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c0cd160 msgr2=0x7f460c0cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:44.585 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.577+0000 7f46140c6700 1 --2- 172.21.15.55:0/2297016159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c0cd160 0x7f460c0cd5e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f460c06ded0 tx=0x7f4604009840 comp rx=0 tx=0).stop 2023-06-23T00:35:44.585 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.577+0000 7f46140c6700 1 -- 172.21.15.55:0/2297016159 shutdown_connections 2023-06-23T00:35:44.586 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.577+0000 7f46140c6700 1 --2- 172.21.15.55:0/2297016159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c0cd160 0x7f460c0cd5e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:44.586 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.577+0000 7f46140c6700 1 --2- 172.21.15.55:0/2297016159 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f460c0b1e90 0x7f460c0b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:44.586 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.577+0000 7f46140c6700 1 --2- 172.21.15.55:0/2297016159 >> v2:172.21.15.141:3300/0 conn(0x7f460c157c00 0x7f460c06eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:44.586 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.577+0000 7f46140c6700 1 -- 172.21.15.55:0/2297016159 >> 172.21.15.55:0/2297016159 conn(0x7f460c069740 msgr2=0x7f460c06a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:44.586 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.577+0000 7f46140c6700 1 -- 172.21.15.55:0/2297016159 shutdown_connections 2023-06-23T00:35:44.586 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.577+0000 7f46140c6700 1 -- 172.21.15.55:0/2297016159 wait complete. 2023-06-23T00:35:44.586 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.578+0000 7f46140c6700 1 Processor -- start 2023-06-23T00:35:44.587 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.578+0000 7f46140c6700 1 -- start start 2023-06-23T00:35:44.587 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f46140c6700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f460c0b1e90 0x7f460c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:44.587 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f46140c6700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f460c0cd160 0x7f460c0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:44.587 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f46140c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c157c00 0x7f460c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:44.587 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f46140c6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c144490 con 0x7f460c157c00 2023-06-23T00:35:44.587 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f46140c6700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0a6010 con 0x7f460c0b1e90 2023-06-23T00:35:44.587 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f46140c6700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c143880 con 0x7f460c0cd160 2023-06-23T00:35:44.589 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4612663700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c157c00 0x7f460c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:44.589 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4612663700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c157c00 0x7f460c0aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50800/0 (socket says 172.21.15.55:50800) 2023-06-23T00:35:44.589 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4612663700 1 -- 172.21.15.55:0/3380811085 learned_addr learned my addr 172.21.15.55:0/3380811085 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:44.589 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4611661700 1 --2- 172.21.15.55:0/3380811085 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f460c0cd160 0x7f460c0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:44.589 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4611e62700 1 --2- 172.21.15.55:0/3380811085 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f460c0b1e90 0x7f460c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:44.590 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4612663700 1 -- 172.21.15.55:0/3380811085 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f460c0cd160 msgr2=0x7f460c0ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:44.590 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4612663700 1 --2- 172.21.15.55:0/3380811085 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f460c0cd160 0x7f460c0ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:44.590 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4612663700 1 -- 172.21.15.55:0/3380811085 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f460c0b1e90 msgr2=0x7f460c0a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:44.590 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4612663700 1 --2- 172.21.15.55:0/3380811085 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f460c0b1e90 0x7f460c0a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:44.590 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4612663700 1 -- 172.21.15.55:0/3380811085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604009ae0 con 0x7f460c157c00 2023-06-23T00:35:44.590 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4612663700 1 --2- 172.21.15.55:0/3380811085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c157c00 0x7f460c0aeda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f460c0675c0 tx=0x7f4604018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:44.590 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4601ffb700 1 -- 172.21.15.55:0/3380811085 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604008ad0 con 0x7f460c157c00 2023-06-23T00:35:44.591 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4601ffb700 1 -- 172.21.15.55:0/3380811085 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4604008c50 con 0x7f460c157c00 2023-06-23T00:35:44.591 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f46140c6700 1 -- 172.21.15.55:0/3380811085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c143b60 con 0x7f460c157c00 2023-06-23T00:35:44.591 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.579+0000 7f4601ffb700 1 -- 172.21.15.55:0/3380811085 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604026460 con 0x7f460c157c00 2023-06-23T00:35:44.591 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.580+0000 7f46140c6700 1 -- 172.21.15.55:0/3380811085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c144120 con 0x7f460c157c00 2023-06-23T00:35:44.591 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.580+0000 7f4601ffb700 1 -- 172.21.15.55:0/3380811085 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4604003040 con 0x7f460c157c00 2023-06-23T00:35:44.591 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.580+0000 7f4601ffb700 1 -- 172.21.15.55:0/3380811085 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f4604026c60 con 0x7f460c157c00 2023-06-23T00:35:44.591 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.580+0000 7f46140c6700 1 -- 172.21.15.55:0/3380811085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c144600 con 0x7f460c157c00 2023-06-23T00:35:44.592 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.583+0000 7f4601ffb700 1 -- 172.21.15.55:0/3380811085 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f4604021070 con 0x7f460c157c00 2023-06-23T00:35:44.621 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:44.618+0000 7fa57af57180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T00:35:44.656 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.652+0000 7f46140c6700 1 -- 172.21.15.55:0/3380811085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f460c04c1b0 con 0x7f460c157c00 2023-06-23T00:35:44.656 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.653+0000 7f4601ffb700 1 -- 172.21.15.55:0/3380811085 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f4604026850 con 0x7f460c157c00 2023-06-23T00:35:44.657 INFO:teuthology.misc.health.smithi055.stdout: 2023-06-23T00:35:44.657 INFO:teuthology.misc.health.smithi055.stdout:{"epoch":10,"fsid":"c0a83a0b-dd71-486d-bdc4-a203f7ef404f","created":"2023-06-23T00:35:40.890983+0000","modified":"2023-06-23T00:35:43.351694+0000","last_up_change":"0.000000","last_in_change":"2023-06-23T00:35:43.351694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d4590eaf-cc24-4ff1-8f1b-9637138bd300","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"518b1bc8-fb1c-405f-8346-fda7978829a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"636a04f1-2379-43ca-9f02-0ee33d2af90a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b893aff1-9f4d-42ca-9af6-42fd6ba3d92c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"259d9777-c081-4c0b-9d83-46d348872e01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"56cfcc58-3f4d-4637-ae5d-a2a52dbdc37f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"49f9b91b-16c2-4394-928e-97446f5feea7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e9806e67-d1e0-49c5-a3e4-898bd4f840e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T00:35:44.657 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.653+0000 7f46140c6700 1 -- 172.21.15.55:0/3380811085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c157c00 msgr2=0x7f460c0aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:44.658 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.653+0000 7f46140c6700 1 --2- 172.21.15.55:0/3380811085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c157c00 0x7f460c0aeda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f460c0675c0 tx=0x7f4604018000 comp rx=0 tx=0).stop 2023-06-23T00:35:44.658 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.654+0000 7f46140c6700 1 -- 172.21.15.55:0/3380811085 shutdown_connections 2023-06-23T00:35:44.658 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.654+0000 7f46140c6700 1 --2- 172.21.15.55:0/3380811085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f460c157c00 0x7f460c0aeda0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:44.658 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.654+0000 7f46140c6700 1 --2- 172.21.15.55:0/3380811085 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f460c0b1e90 0x7f460c0a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:44.658 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.654+0000 7f46140c6700 1 --2- 172.21.15.55:0/3380811085 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f460c0cd160 0x7f460c0ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:44.658 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.654+0000 7f46140c6700 1 -- 172.21.15.55:0/3380811085 >> 172.21.15.55:0/3380811085 conn(0x7f460c069740 msgr2=0x7f460c0b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:44.659 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.654+0000 7f46140c6700 1 -- 172.21.15.55:0/3380811085 shutdown_connections 2023-06-23T00:35:44.659 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:44.654+0000 7f46140c6700 1 -- 172.21.15.55:0/3380811085 wait complete. 2023-06-23T00:35:44.667 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:44.663+0000 7fde1d809180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T00:35:44.668 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-06-23T00:35:44.871 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:44.868+0000 7fa57af57180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T00:35:44.925 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:44.921+0000 7fde1d809180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T00:35:44.948 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:44.945+0000 7fa57af57180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T00:35:45.006 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:45.002+0000 7fde1d809180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T00:35:45.771 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:45.768+0000 7fa57af57180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T00:35:45.844 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:45.841+0000 7fa57af57180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T00:35:45.891 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:45.887+0000 7fde1d809180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T00:35:45.919 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:45.916+0000 7fa57af57180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T00:35:45.968 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:45.964+0000 7fde1d809180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T00:35:46.046 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:46.042+0000 7fde1d809180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T00:35:46.052 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:46.049+0000 7fa57af57180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T00:35:46.136 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:46.133+0000 7fa57af57180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T00:35:46.186 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:46.182+0000 7fde1d809180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T00:35:46.275 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:46.271+0000 7fde1d809180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T00:35:46.290 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:46.287+0000 7fa57af57180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T00:35:46.366 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:46.363+0000 7fa57af57180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T00:35:46.435 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:46.431+0000 7fde1d809180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T00:35:46.440 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:46.437+0000 7fa57af57180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T00:35:46.521 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:46.517+0000 7fde1d809180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T00:35:46.524 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:46.521+0000 7fa57af57180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T00:35:46.604 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:46.601+0000 7fde1d809180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T00:35:46.684 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-06-23T00:35:46.680+0000 7fde1d809180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T00:35:47.269 INFO:tasks.ceph.osd.5.smithi141.stderr:2023-06-23T00:35:47.265+0000 7f4c7002e540 -1 osd.5 0 log_to_monitors true 2023-06-23T00:35:47.277 INFO:tasks.ceph.osd.7.smithi141.stderr:2023-06-23T00:35:47.272+0000 7f6fdf5c8540 -1 osd.7 0 log_to_monitors true 2023-06-23T00:35:47.459 INFO:tasks.ceph.osd.2.smithi055.stderr:2023-06-23T00:35:47.455+0000 7ffb6dd40540 -1 osd.2 0 log_to_monitors true 2023-06-23T00:35:47.500 INFO:tasks.ceph.osd.1.smithi055.stderr:2023-06-23T00:35:47.496+0000 7f9d4e098540 -1 osd.1 0 log_to_monitors true 2023-06-23T00:35:47.502 INFO:tasks.ceph.osd.6.smithi141.stderr:2023-06-23T00:35:47.498+0000 7f9ab6348540 -1 osd.6 0 log_to_monitors true 2023-06-23T00:35:47.523 INFO:tasks.ceph.osd.0.smithi055.stderr:2023-06-23T00:35:47.520+0000 7f9634bfb540 -1 osd.0 0 log_to_monitors true 2023-06-23T00:35:47.528 INFO:tasks.ceph.osd.3.smithi055.stderr:2023-06-23T00:35:47.524+0000 7fb8ba8ae540 -1 osd.3 0 log_to_monitors true 2023-06-23T00:35:47.544 INFO:tasks.ceph.osd.4.smithi141.stderr:2023-06-23T00:35:47.540+0000 7f7d28a8f540 -1 osd.4 0 log_to_monitors true 2023-06-23T00:35:47.550 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:47.546+0000 7fa534358700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-06-23T00:35:47.550 INFO:tasks.ceph.mgr.x.smithi141.stderr:2023-06-23T00:35:47.547+0000 7fa534358700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-06-23T00:35:50.150 INFO:tasks.ceph.osd.1.smithi055.stderr:2023-06-23T00:35:50.147+0000 7f9d44cba700 -1 osd.1 0 waiting for initial osdmap 2023-06-23T00:35:50.157 INFO:tasks.ceph.osd.2.smithi055.stderr:2023-06-23T00:35:50.153+0000 7ffb64962700 -1 osd.2 0 waiting for initial osdmap 2023-06-23T00:35:50.160 INFO:tasks.ceph.osd.1.smithi055.stderr:2023-06-23T00:35:50.156+0000 7f9d3dab1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T00:35:50.162 INFO:tasks.ceph.osd.2.smithi055.stderr:2023-06-23T00:35:50.158+0000 7ffb5d759700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T00:35:50.163 INFO:tasks.ceph.osd.0.smithi055.stderr:2023-06-23T00:35:50.160+0000 7f962b81d700 -1 osd.0 0 waiting for initial osdmap 2023-06-23T00:35:50.165 INFO:tasks.ceph.osd.3.smithi055.stderr:2023-06-23T00:35:50.162+0000 7fb8b14d0700 -1 osd.3 0 waiting for initial osdmap 2023-06-23T00:35:50.170 INFO:tasks.ceph.osd.3.smithi055.stderr:2023-06-23T00:35:50.167+0000 7fb8aa2c7700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T00:35:50.178 INFO:tasks.ceph.osd.0.smithi055.stderr:2023-06-23T00:35:50.173+0000 7f9624614700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T00:35:50.745 INFO:tasks.ceph.osd.7.smithi141.stderr:2023-06-23T00:35:50.742+0000 7f6fd61ea700 -1 osd.7 0 waiting for initial osdmap 2023-06-23T00:35:50.752 INFO:tasks.ceph.osd.7.smithi141.stderr:2023-06-23T00:35:50.746+0000 7f6fcefe1700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T00:35:50.767 INFO:tasks.ceph.osd.4.smithi141.stderr:2023-06-23T00:35:50.763+0000 7f7d1f6b1700 -1 osd.4 0 waiting for initial osdmap 2023-06-23T00:35:50.771 INFO:tasks.ceph.osd.4.smithi141.stderr:2023-06-23T00:35:50.768+0000 7f7d184a8700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T00:35:50.772 INFO:tasks.ceph.osd.6.smithi141.stderr:2023-06-23T00:35:50.769+0000 7f9aacf6a700 -1 osd.6 0 waiting for initial osdmap 2023-06-23T00:35:50.777 INFO:tasks.ceph.osd.6.smithi141.stderr:2023-06-23T00:35:50.774+0000 7f9aa5d61700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T00:35:50.782 INFO:tasks.ceph.osd.5.smithi141.stderr:2023-06-23T00:35:50.779+0000 7f4c66c50700 -1 osd.5 0 waiting for initial osdmap 2023-06-23T00:35:50.785 INFO:tasks.ceph.osd.5.smithi141.stderr:2023-06-23T00:35:50.782+0000 7f4c5fa47700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T00:35:51.474 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-23T00:35:51.599 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.595+0000 7f1dce444700 1 Processor -- start 2023-06-23T00:35:51.599 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.595+0000 7f1dce444700 1 -- start start 2023-06-23T00:35:51.601 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dce444700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f1dc80cafa0 0x7f1dc80c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.601 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dce444700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1dc8158710 0x7f1dc8156d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.601 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dce444700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1dc80c8ed0 0x7f1dc81572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.602 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dce444700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc815bef0 con 0x7f1dc80c8ed0 2023-06-23T00:35:51.602 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dce444700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc80c9350 con 0x7f1dc80cafa0 2023-06-23T00:35:51.602 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dce444700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc8157810 con 0x7f1dc8158710 2023-06-23T00:35:51.602 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dcc9e1700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1dc80c8ed0 0x7f1dc81572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.602 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dcc9e1700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1dc80c8ed0 0x7f1dc81572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39354/0 (socket says 172.21.15.55:39354) 2023-06-23T00:35:51.602 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dcc9e1700 1 -- 172.21.15.55:0/3388603888 learned_addr learned my addr 172.21.15.55:0/3388603888 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:51.602 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dc77fe700 1 --2- 172.21.15.55:0/3388603888 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1dc8158710 0x7f1dc8156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.602 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dc7fff700 1 --2- 172.21.15.55:0/3388603888 >> v2:172.21.15.141:3300/0 conn(0x7f1dc80cafa0 0x7f1dc80c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.603 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dcc9e1700 1 -- 172.21.15.55:0/3388603888 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1dc8158710 msgr2=0x7f1dc8156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:51.603 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dcc9e1700 1 --2- 172.21.15.55:0/3388603888 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1dc8158710 0x7f1dc8156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.603 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dcc9e1700 1 -- 172.21.15.55:0/3388603888 >> v2:172.21.15.141:3300/0 conn(0x7f1dc80cafa0 msgr2=0x7f1dc80c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:51.603 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dcc9e1700 1 --2- 172.21.15.55:0/3388603888 >> v2:172.21.15.141:3300/0 conn(0x7f1dc80cafa0 0x7f1dc80c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.603 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dcc9e1700 1 -- 172.21.15.55:0/3388603888 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc815b130 con 0x7f1dc80c8ed0 2023-06-23T00:35:51.603 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dc7fff700 1 --2- 172.21.15.55:0/3388603888 >> v2:172.21.15.141:3300/0 conn(0x7f1dc80cafa0 0x7f1dc80c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T00:35:51.603 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.596+0000 7f1dcc9e1700 1 --2- 172.21.15.55:0/3388603888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc80c8ed0 0x7f1dc81572d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009f20 tx=0x7f1dc0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ed6f1faa82491b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:51.604 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dc6ffd700 1 -- 172.21.15.55:0/3388603888 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc000e070 con 0x7f1dc80c8ed0 2023-06-23T00:35:51.604 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dc6ffd700 1 -- 172.21.15.55:0/3388603888 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc000ba40 con 0x7f1dc80c8ed0 2023-06-23T00:35:51.604 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dc6ffd700 1 -- 172.21.15.55:0/3388603888 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00092e0 con 0x7f1dc80c8ed0 2023-06-23T00:35:51.604 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dce444700 1 -- 172.21.15.55:0/3388603888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc80c8ed0 msgr2=0x7f1dc81572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:51.604 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dce444700 1 --2- 172.21.15.55:0/3388603888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc80c8ed0 0x7f1dc81572d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009f20 tx=0x7f1dc0009840 comp rx=0 tx=0).stop 2023-06-23T00:35:51.604 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dce444700 1 -- 172.21.15.55:0/3388603888 shutdown_connections 2023-06-23T00:35:51.604 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dce444700 1 --2- 172.21.15.55:0/3388603888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc80c8ed0 0x7f1dc81572d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.605 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dce444700 1 --2- 172.21.15.55:0/3388603888 >> v2:172.21.15.141:3300/0 conn(0x7f1dc80cafa0 0x7f1dc80c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.605 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dce444700 1 --2- 172.21.15.55:0/3388603888 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1dc8158710 0x7f1dc8156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.605 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dce444700 1 -- 172.21.15.55:0/3388603888 >> 172.21.15.55:0/3388603888 conn(0x7f1dc806b530 msgr2=0x7f1dc806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:51.605 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dce444700 1 -- 172.21.15.55:0/3388603888 shutdown_connections 2023-06-23T00:35:51.605 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.597+0000 7f1dce444700 1 -- 172.21.15.55:0/3388603888 wait complete. 2023-06-23T00:35:51.607 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.598+0000 7f1dce444700 1 Processor -- start 2023-06-23T00:35:51.607 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.599+0000 7f1dce444700 1 -- start start 2023-06-23T00:35:51.607 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.599+0000 7f1dce444700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1dc80cafa0 0x7f1dc8169190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.607 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.599+0000 7f1dce444700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc8158710 0x7f1dc80c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.608 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.599+0000 7f1dce444700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1dc80b3110 0x7f1dc80b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.608 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.599+0000 7f1dce444700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc816a770 con 0x7f1dc8158710 2023-06-23T00:35:51.608 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.599+0000 7f1dce444700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc8169ae0 con 0x7f1dc80b3110 2023-06-23T00:35:51.608 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.599+0000 7f1dce444700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc8169dc0 con 0x7f1dc80cafa0 2023-06-23T00:35:51.608 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.599+0000 7f1dc77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc8158710 0x7f1dc80c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.608 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dc77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc8158710 0x7f1dc80c33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39356/0 (socket says 172.21.15.55:39356) 2023-06-23T00:35:51.608 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dc77fe700 1 -- 172.21.15.55:0/107267583 learned_addr learned my addr 172.21.15.55:0/107267583 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:51.609 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dcc9e1700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1dc80b3110 0x7f1dc80b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.609 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dc7fff700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1dc80cafa0 0x7f1dc8169190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.609 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dc77fe700 1 -- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1dc80cafa0 msgr2=0x7f1dc8169190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:51.609 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dc77fe700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1dc80cafa0 0x7f1dc8169190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.609 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dc77fe700 1 -- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1dc80b3110 msgr2=0x7f1dc80b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:51.609 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dc77fe700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1dc80b3110 0x7f1dc80b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.609 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dc77fe700 1 -- 172.21.15.55:0/107267583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc0009ae0 con 0x7f1dc8158710 2023-06-23T00:35:51.609 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dc77fe700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc8158710 0x7f1dc80c33b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1db000d860 tx=0x7f1db0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:51.610 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dbffff700 1 -- 172.21.15.55:0/107267583 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00125f0 con 0x7f1dc8158710 2023-06-23T00:35:51.610 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dbffff700 1 -- 172.21.15.55:0/107267583 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db0012750 con 0x7f1dc8158710 2023-06-23T00:35:51.610 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dbffff700 1 -- 172.21.15.55:0/107267583 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0012b10 con 0x7f1dc8158710 2023-06-23T00:35:51.610 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dce444700 1 -- 172.21.15.55:0/107267583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc816a0a0 con 0x7f1dc8158710 2023-06-23T00:35:51.610 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.600+0000 7f1dce444700 1 -- 172.21.15.55:0/107267583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc81696d0 con 0x7f1dc8158710 2023-06-23T00:35:51.610 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.602+0000 7f1dce444700 1 -- 172.21.15.55:0/107267583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc804c1b0 con 0x7f1dc8158710 2023-06-23T00:35:51.610 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.602+0000 7f1dbffff700 1 -- 172.21.15.55:0/107267583 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7f1db001e460 con 0x7f1dc8158710 2023-06-23T00:35:51.610 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.602+0000 7f1dbffff700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1da8066e00 0x7f1da80692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.611 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.602+0000 7f1dbffff700 1 -- 172.21.15.55:0/107267583 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f1db0084350 con 0x7f1dc8158710 2023-06-23T00:35:51.611 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.603+0000 7f1dc7fff700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1da8066e00 0x7f1da80692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.611 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.603+0000 7f1dc7fff700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1da8066e00 0x7f1da80692b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1db800ad40 tx=0x7f1db8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:51.611 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.607+0000 7f1dbffff700 1 -- 172.21.15.55:0/107267583 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f1db0051c30 con 0x7f1dc8158710 2023-06-23T00:35:51.809 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.806+0000 7f1dce444700 1 -- 172.21.15.55:0/107267583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dc816a8b0 con 0x7f1dc8158710 2023-06-23T00:35:51.810 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.806+0000 7f1dbffff700 1 -- 172.21.15.55:0/107267583 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10396 (secure 0 0 0) 0x7f1db0055250 con 0x7f1dc8158710 2023-06-23T00:35:51.810 INFO:teuthology.misc.health.smithi055.stdout: 2023-06-23T00:35:51.811 INFO:teuthology.misc.health.smithi055.stdout:{"epoch":13,"fsid":"c0a83a0b-dd71-486d-bdc4-a203f7ef404f","created":"2023-06-23T00:35:40.890983+0000","modified":"2023-06-23T00:35:51.173985+0000","last_up_change":"2023-06-23T00:35:51.173985+0000","last_in_change":"2023-06-23T00:35:43.351694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d4590eaf-cc24-4ff1-8f1b-9637138bd300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":87956},{"type":"v1","addr":"172.21.15.55:6817","nonce":87956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":87956},{"type":"v1","addr":"172.21.15.55:6819","nonce":87956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":87956},{"type":"v1","addr":"172.21.15.55:6823","nonce":87956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":87956},{"type":"v1","addr":"172.21.15.55:6821","nonce":87956}]},"public_addr":"172.21.15.55:6817/87956","cluster_addr":"172.21.15.55:6819/87956","heartbeat_back_addr":"172.21.15.55:6823/87956","heartbeat_front_addr":"172.21.15.55:6821/87956","state":["exists","up"]},{"osd":1,"uuid":"518b1bc8-fb1c-405f-8346-fda7978829a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":87959},{"type":"v1","addr":"172.21.15.55:6809","nonce":87959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":87959},{"type":"v1","addr":"172.21.15.55:6811","nonce":87959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":87959},{"type":"v1","addr":"172.21.15.55:6815","nonce":87959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":87959},{"type":"v1","addr":"172.21.15.55:6813","nonce":87959}]},"public_addr":"172.21.15.55:6809/87959","cluster_addr":"172.21.15.55:6811/87959","heartbeat_back_addr":"172.21.15.55:6815/87959","heartbeat_front_addr":"172.21.15.55:6813/87959","state":["exists","up"]},{"osd":2,"uuid":"636a04f1-2379-43ca-9f02-0ee33d2af90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":87955},{"type":"v1","addr":"172.21.15.55:6801","nonce":87955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":87955},{"type":"v1","addr":"172.21.15.55:6803","nonce":87955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":87955},{"type":"v1","addr":"172.21.15.55:6807","nonce":87955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":87955},{"type":"v1","addr":"172.21.15.55:6805","nonce":87955}]},"public_addr":"172.21.15.55:6801/87955","cluster_addr":"172.21.15.55:6803/87955","heartbeat_back_addr":"172.21.15.55:6807/87955","heartbeat_front_addr":"172.21.15.55:6805/87955","state":["exists","up"]},{"osd":3,"uuid":"b893aff1-9f4d-42ca-9af6-42fd6ba3d92c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":87972},{"type":"v1","addr":"172.21.15.55:6825","nonce":87972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":87972},{"type":"v1","addr":"172.21.15.55:6827","nonce":87972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":87972},{"type":"v1","addr":"172.21.15.55:6831","nonce":87972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":87972},{"type":"v1","addr":"172.21.15.55:6829","nonce":87972}]},"public_addr":"172.21.15.55:6825/87972","cluster_addr":"172.21.15.55:6827/87972","heartbeat_back_addr":"172.21.15.55:6831/87972","heartbeat_front_addr":"172.21.15.55:6829/87972","state":["exists","up"]},{"osd":4,"uuid":"259d9777-c081-4c0b-9d83-46d348872e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":88177},{"type":"v1","addr":"172.21.15.141:6817","nonce":88177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":88177},{"type":"v1","addr":"172.21.15.141:6819","nonce":88177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6825","nonce":88177},{"type":"v1","addr":"172.21.15.141:6827","nonce":88177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6821","nonce":88177},{"type":"v1","addr":"172.21.15.141:6823","nonce":88177}]},"public_addr":"172.21.15.141:6817/88177","cluster_addr":"172.21.15.141:6819/88177","heartbeat_back_addr":"172.21.15.141:6827/88177","heartbeat_front_addr":"172.21.15.141:6823/88177","state":["exists","up"]},{"osd":5,"uuid":"56cfcc58-3f4d-4637-ae5d-a2a52dbdc37f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":88161},{"type":"v1","addr":"172.21.15.141:6801","nonce":88161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":88161},{"type":"v1","addr":"172.21.15.141:6803","nonce":88161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":88161},{"type":"v1","addr":"172.21.15.141:6807","nonce":88161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":88161},{"type":"v1","addr":"172.21.15.141:6805","nonce":88161}]},"public_addr":"172.21.15.141:6801/88161","cluster_addr":"172.21.15.141:6803/88161","heartbeat_back_addr":"172.21.15.141:6807/88161","heartbeat_front_addr":"172.21.15.141:6805/88161","state":["exists","up"]},{"osd":6,"uuid":"49f9b91b-16c2-4394-928e-97446f5feea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":88192},{"type":"v1","addr":"172.21.15.141:6822","nonce":88192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":88192},{"type":"v1","addr":"172.21.15.141:6826","nonce":88192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":88192},{"type":"v1","addr":"172.21.15.141:6831","nonce":88192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":88192},{"type":"v1","addr":"172.21.15.141:6829","nonce":88192}]},"public_addr":"172.21.15.141:6822/88192","cluster_addr":"172.21.15.141:6826/88192","heartbeat_back_addr":"172.21.15.141:6831/88192","heartbeat_front_addr":"172.21.15.141:6829/88192","state":["exists","up"]},{"osd":7,"uuid":"e9806e67-d1e0-49c5-a3e4-898bd4f840e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":88162},{"type":"v1","addr":"172.21.15.141:6809","nonce":88162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":88162},{"type":"v1","addr":"172.21.15.141:6811","nonce":88162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":88162},{"type":"v1","addr":"172.21.15.141:6815","nonce":88162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":88162},{"type":"v1","addr":"172.21.15.141:6813","nonce":88162}]},"public_addr":"172.21.15.141:6809/88162","cluster_addr":"172.21.15.141:6811/88162","heartbeat_back_addr":"172.21.15.141:6815/88162","heartbeat_front_addr":"172.21.15.141:6813/88162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T00:35:51.813 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 -- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1da8066e00 msgr2=0x7f1da80692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:51.813 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1da8066e00 0x7f1da80692b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1db800ad40 tx=0x7f1db8008040 comp rx=0 tx=0).stop 2023-06-23T00:35:51.814 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 -- 172.21.15.55:0/107267583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc8158710 msgr2=0x7f1dc80c33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:51.814 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc8158710 0x7f1dc80c33b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1db000d860 tx=0x7f1db0008040 comp rx=0 tx=0).stop 2023-06-23T00:35:51.814 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 -- 172.21.15.55:0/107267583 shutdown_connections 2023-06-23T00:35:51.815 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1da8066e00 0x7f1da80692b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.815 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1dc80b3110 0x7f1dc80b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.815 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1dc80cafa0 0x7f1dc8169190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.816 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 --2- 172.21.15.55:0/107267583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dc8158710 0x7f1dc80c33b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.816 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 -- 172.21.15.55:0/107267583 >> 172.21.15.55:0/107267583 conn(0x7f1dc806b530 msgr2=0x7f1dc80cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:51.817 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 -- 172.21.15.55:0/107267583 shutdown_connections 2023-06-23T00:35:51.817 INFO:teuthology.misc.health.smithi055.stderr:2023-06-23T00:35:51.809+0000 7f1dce444700 1 -- 172.21.15.55:0/107267583 wait complete. 2023-06-23T00:35:51.824 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-06-23T00:35:51.825 INFO:tasks.ceph:Creating RBD pool 2023-06-23T00:35:51.825 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd pool create rbd 8 2023-06-23T00:35:51.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.941+0000 7fd288837700 1 Processor -- start 2023-06-23T00:35:51.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.941+0000 7fd288837700 1 -- start start 2023-06-23T00:35:51.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.941+0000 7fd288837700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd284153670 0x7fd28415a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.941+0000 7fd288837700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7fd284153a40 0x7fd2840affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.941+0000 7fd288837700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2840b0510 con 0x7fd284067410 2023-06-23T00:35:51.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.941+0000 7fd288837700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2840af560 con 0x7fd284153670 2023-06-23T00:35:51.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.941+0000 7fd288837700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2840af840 con 0x7fd284153a40 2023-06-23T00:35:51.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2837fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd284153670 0x7fd28415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2837fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd284153670 0x7fd28415a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39364/0 (socket says 172.21.15.55:39364) 2023-06-23T00:35:51.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2837fe700 1 -- 172.21.15.55:0/3634510481 learned_addr learned my addr 172.21.15.55:0/3634510481 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:51.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd282ffd700 1 --2- 172.21.15.55:0/3634510481 >> v2:172.21.15.141:3300/0 conn(0x7fd284153a40 0x7fd2840affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2837fe700 1 -- 172.21.15.55:0/3634510481 >> v2:172.21.15.141:3300/0 conn(0x7fd284153a40 msgr2=0x7fd2840affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2837fe700 1 --2- 172.21.15.55:0/3634510481 >> v2:172.21.15.141:3300/0 conn(0x7fd284153a40 0x7fd2840affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2837fe700 1 -- 172.21.15.55:0/3634510481 >> v1:172.21.15.141:6789/0 conn(0x7fd284067410 legacy=0x7fd28415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2837fe700 1 -- 172.21.15.55:0/3634510481 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2840afb20 con 0x7fd284153670 2023-06-23T00:35:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2837fe700 1 --2- 172.21.15.55:0/3634510481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284153670 0x7fd28415a680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd274009f20 tx=0x7fd274009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dfd6e0dfab502e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2827fc700 1 -- 172.21.15.55:0/3634510481 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27400e070 con 0x7fd284153670 2023-06-23T00:35:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2827fc700 1 -- 172.21.15.55:0/3634510481 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27400b990 con 0x7fd284153670 2023-06-23T00:35:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.942+0000 7fd2827fc700 1 -- 172.21.15.55:0/3634510481 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2740092e0 con 0x7fd284153670 2023-06-23T00:35:51.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.943+0000 7fd288837700 1 -- 172.21.15.55:0/3634510481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284153670 msgr2=0x7fd28415a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:51.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.943+0000 7fd288837700 1 --2- 172.21.15.55:0/3634510481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284153670 0x7fd28415a680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd274009f20 tx=0x7fd274009840 comp rx=0 tx=0).stop 2023-06-23T00:35:51.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.943+0000 7fd288837700 1 -- 172.21.15.55:0/3634510481 shutdown_connections 2023-06-23T00:35:51.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.943+0000 7fd288837700 1 --2- 172.21.15.55:0/3634510481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284153670 0x7fd28415a680 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.943+0000 7fd288837700 1 --2- 172.21.15.55:0/3634510481 >> v2:172.21.15.141:3300/0 conn(0x7fd284153a40 0x7fd2840affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.943+0000 7fd288837700 1 -- 172.21.15.55:0/3634510481 >> 172.21.15.55:0/3634510481 conn(0x7fd28406aa80 msgr2=0x7fd2840aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:51.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.943+0000 7fd288837700 1 -- 172.21.15.55:0/3634510481 shutdown_connections 2023-06-23T00:35:51.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.943+0000 7fd288837700 1 -- 172.21.15.55:0/3634510481 wait complete. 2023-06-23T00:35:51.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.944+0000 7fd288837700 1 Processor -- start 2023-06-23T00:35:51.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.944+0000 7fd288837700 1 -- start start 2023-06-23T00:35:51.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd288837700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284067410 0x7fd28414b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd288837700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fd284153670 0x7fd284151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd288837700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd284153a40 0x7fd2840ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd288837700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284151f40 con 0x7fd284067410 2023-06-23T00:35:51.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd288837700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284154660 con 0x7fd284153a40 2023-06-23T00:35:51.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd288837700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd284151c60 con 0x7fd284153670 2023-06-23T00:35:51.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd2837fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284067410 0x7fd28414b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd2837fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284067410 0x7fd28414b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39372/0 (socket says 172.21.15.55:39372) 2023-06-23T00:35:51.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd2837fe700 1 -- 172.21.15.55:0/2527320677 learned_addr learned my addr 172.21.15.55:0/2527320677 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:51.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd282ffd700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fd284153670 0x7fd284151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd283fff700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd284153a40 0x7fd2840ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd2837fe700 1 -- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fd284153670 msgr2=0x7fd284151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:51.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd2837fe700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fd284153670 0x7fd284151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd2837fe700 1 -- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd284153a40 msgr2=0x7fd2840ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:51.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd2837fe700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd284153a40 0x7fd2840ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:51.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd2837fe700 1 -- 172.21.15.55:0/2527320677 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274009ae0 con 0x7fd284067410 2023-06-23T00:35:51.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd2837fe700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284067410 0x7fd28414b0d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd274009880 tx=0x7fd274018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:51.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd27b7fe700 1 -- 172.21.15.55:0/2527320677 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274008820 con 0x7fd284067410 2023-06-23T00:35:51.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd27b7fe700 1 -- 172.21.15.55:0/2527320677 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd274008980 con 0x7fd284067410 2023-06-23T00:35:51.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd288837700 1 -- 172.21.15.55:0/2527320677 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd284154010 con 0x7fd284067410 2023-06-23T00:35:51.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.945+0000 7fd27b7fe700 1 -- 172.21.15.55:0/2527320677 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274008d20 con 0x7fd284067410 2023-06-23T00:35:51.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.946+0000 7fd288837700 1 -- 172.21.15.55:0/2527320677 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2841518a0 con 0x7fd284067410 2023-06-23T00:35:51.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.946+0000 7fd288837700 1 -- 172.21.15.55:0/2527320677 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd284154940 con 0x7fd284067410 2023-06-23T00:35:51.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.947+0000 7fd27b7fe700 1 -- 172.21.15.55:0/2527320677 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7fd274003040 con 0x7fd284067410 2023-06-23T00:35:51.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.947+0000 7fd27b7fe700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fd264066e00 0x7fd2640692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:51.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.947+0000 7fd27b7fe700 1 -- 172.21.15.55:0/2527320677 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fd2740859b0 con 0x7fd284067410 2023-06-23T00:35:51.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.947+0000 7fd282ffd700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fd264066e00 0x7fd2640692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:51.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.948+0000 7fd282ffd700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fd264066e00 0x7fd2640692b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd2840ce210 tx=0x7fd26c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:51.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:51.952+0000 7fd27b7fe700 1 -- 172.21.15.55:0/2527320677 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fd274053290 con 0x7fd284067410 2023-06-23T00:35:52.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.146+0000 7fd288837700 1 -- 172.21.15.55:0/2527320677 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fd284159ba0 con 0x7fd284067410 2023-06-23T00:35:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.193+0000 7fd27b7fe700 1 -- 172.21.15.55:0/2527320677 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7fd2740568b0 con 0x7fd284067410 2023-06-23T00:35:52.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 -- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fd264066e00 msgr2=0x7fd2640692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:52.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fd264066e00 0x7fd2640692b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd2840ce210 tx=0x7fd26c0091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 -- 172.21.15.55:0/2527320677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284067410 msgr2=0x7fd28414b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:52.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284067410 0x7fd28414b0d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd274009880 tx=0x7fd274018000 comp rx=0 tx=0).stop 2023-06-23T00:35:52.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 -- 172.21.15.55:0/2527320677 shutdown_connections 2023-06-23T00:35:52.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fd264066e00 0x7fd2640692b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fd284153a40 0x7fd2840ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd284067410 0x7fd28414b0d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 --2- 172.21.15.55:0/2527320677 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fd284153670 0x7fd284151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 -- 172.21.15.55:0/2527320677 >> 172.21.15.55:0/2527320677 conn(0x7fd28406aa80 msgr2=0x7fd2840acc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:52.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 -- 172.21.15.55:0/2527320677 shutdown_connections 2023-06-23T00:35:52.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.196+0000 7fd288837700 1 -- 172.21.15.55:0/2527320677 wait complete. 2023-06-23T00:35:52.202 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'rbd' created 2023-06-23T00:35:52.212 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-06-23T00:35:52.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.331+0000 7f8115bd8700 1 Processor -- start 2023-06-23T00:35:52.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.331+0000 7f8115bd8700 1 -- start start 2023-06-23T00:35:52.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.331+0000 7f8115bd8700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110068190 0x7f8110068560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:52.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.331+0000 7f8115bd8700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f8110067410 0x7f81100ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:52.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.331+0000 7f8115bd8700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81100696d0 con 0x7f811015d090 2023-06-23T00:35:52.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.331+0000 7f8115bd8700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81100cbbd0 con 0x7f8110067410 2023-06-23T00:35:52.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.331+0000 7f8115bd8700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811015d650 con 0x7f8110068190 2023-06-23T00:35:52.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f810ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f811015d090 0x7f811015a510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56874/0 (socket says 172.21.15.55:56874) 2023-06-23T00:35:52.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f810ffff700 1 -- 172.21.15.55:0/4018607149 learned_addr learned my addr 172.21.15.55:0/4018607149 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:52.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f8114bd6700 1 --2- 172.21.15.55:0/4018607149 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110068190 0x7f8110068560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:52.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f81153d7700 1 --2- 172.21.15.55:0/4018607149 >> v2:172.21.15.141:3300/0 conn(0x7f8110067410 0x7f81100ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:52.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f810f7fe700 1 -- 172.21.15.55:0/4018607149 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580688179 0 0) 0x7f81100696d0 con 0x7f811015d090 2023-06-23T00:35:52.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f810f7fe700 1 -- 172.21.15.55:0/4018607149 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80fc003570 con 0x7f811015d090 2023-06-23T00:35:52.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f81153d7700 1 -- 172.21.15.55:0/4018607149 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110068190 msgr2=0x7f8110068560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:52.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f81153d7700 1 --2- 172.21.15.55:0/4018607149 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110068190 0x7f8110068560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f81153d7700 1 -- 172.21.15.55:0/4018607149 >> v1:172.21.15.55:6789/0 conn(0x7f811015d090 legacy=0x7f811015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:52.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f81153d7700 1 -- 172.21.15.55:0/4018607149 --> v2:172.21.15.141:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811015d930 con 0x7f8110067410 2023-06-23T00:35:52.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f8114bd6700 1 --2- 172.21.15.55:0/4018607149 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110068190 0x7f8110068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:52.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f81153d7700 1 --2- 172.21.15.55:0/4018607149 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8110067410 0x7f81100ca470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f81000098e0 tx=0x7f81000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=274549e24ed6ace8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:52.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f810f7fe700 1 -- 172.21.15.55:0/4018607149 <== mon.1 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8100011630 con 0x7f8110067410 2023-06-23T00:35:52.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f810f7fe700 1 -- 172.21.15.55:0/4018607149 <== mon.1 v2:172.21.15.141:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8100011790 con 0x7f8110067410 2023-06-23T00:35:52.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.332+0000 7f810f7fe700 1 -- 172.21.15.55:0/4018607149 <== mon.1 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8100011b10 con 0x7f8110067410 2023-06-23T00:35:52.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.333+0000 7f8115bd8700 1 -- 172.21.15.55:0/4018607149 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8110067410 msgr2=0x7f81100ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:52.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.333+0000 7f8115bd8700 1 --2- 172.21.15.55:0/4018607149 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8110067410 0x7f81100ca470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f81000098e0 tx=0x7f81000091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.333+0000 7f8115bd8700 1 -- 172.21.15.55:0/4018607149 shutdown_connections 2023-06-23T00:35:52.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.333+0000 7f8115bd8700 1 --2- 172.21.15.55:0/4018607149 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8110067410 0x7f81100ca470 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.333+0000 7f8115bd8700 1 --2- 172.21.15.55:0/4018607149 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110068190 0x7f8110068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.333+0000 7f8115bd8700 1 -- 172.21.15.55:0/4018607149 >> 172.21.15.55:0/4018607149 conn(0x7f81100cc8a0 msgr2=0x7f81100ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:52.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.333+0000 7f8115bd8700 1 -- 172.21.15.55:0/4018607149 shutdown_connections 2023-06-23T00:35:52.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.333+0000 7f8115bd8700 1 -- 172.21.15.55:0/4018607149 wait complete. 2023-06-23T00:35:52.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.334+0000 7f8115bd8700 1 Processor -- start 2023-06-23T00:35:52.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.334+0000 7f8115bd8700 1 -- start start 2023-06-23T00:35:52.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f8115bd8700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110067410 0x7f81100ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:52.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f8115bd8700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8110068190 0x7f81100aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:52.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f8115bd8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f811015d090 0x7f81100acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:52.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f8115bd8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100aae60 con 0x7f811015d090 2023-06-23T00:35:52.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f8115bd8700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100aa250 con 0x7f8110068190 2023-06-23T00:35:52.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f8115bd8700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81100aa530 con 0x7f8110067410 2023-06-23T00:35:52.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f81153d7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f811015d090 0x7f81100acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:52.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f81153d7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f811015d090 0x7f81100acfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39386/0 (socket says 172.21.15.55:39386) 2023-06-23T00:35:52.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f81153d7700 1 -- 172.21.15.55:0/2868664548 learned_addr learned my addr 172.21.15.55:0/2868664548 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:52.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f8114bd6700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110067410 0x7f81100ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:52.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f810ffff700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8110068190 0x7f81100aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:52.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f81153d7700 1 -- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110067410 msgr2=0x7f81100ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:52.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f81153d7700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110067410 0x7f81100ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f81153d7700 1 -- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8110068190 msgr2=0x7f81100aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:52.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f81153d7700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8110068190 0x7f81100aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:52.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f81153d7700 1 -- 172.21.15.55:0/2868664548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8108009a20 con 0x7f811015d090 2023-06-23T00:35:52.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.335+0000 7f8114bd6700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110067410 0x7f81100ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:52.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.336+0000 7f81153d7700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f811015d090 0x7f81100acfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f81100cd450 tx=0x7f8100017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:52.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.336+0000 7f810cff9700 1 -- 172.21.15.55:0/2868664548 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810001a410 con 0x7f811015d090 2023-06-23T00:35:52.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.336+0000 7f810cff9700 1 -- 172.21.15.55:0/2868664548 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810001a570 con 0x7f811015d090 2023-06-23T00:35:52.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.336+0000 7f8115bd8700 1 -- 172.21.15.55:0/2868664548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8100009420 con 0x7f811015d090 2023-06-23T00:35:52.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.336+0000 7f810cff9700 1 -- 172.21.15.55:0/2868664548 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810001a930 con 0x7f811015d090 2023-06-23T00:35:52.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.336+0000 7f8115bd8700 1 -- 172.21.15.55:0/2868664548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81100aaaf0 con 0x7f811015d090 2023-06-23T00:35:52.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.337+0000 7f8115bd8700 1 -- 172.21.15.55:0/2868664548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81100aacb0 con 0x7f811015d090 2023-06-23T00:35:52.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.337+0000 7f810cff9700 1 -- 172.21.15.55:0/2868664548 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7f810001aca0 con 0x7f811015d090 2023-06-23T00:35:52.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.338+0000 7f810cff9700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f80f0066e00 0x7f80f00692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:52.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.338+0000 7f810cff9700 1 -- 172.21.15.55:0/2868664548 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4809+0+0 (secure 0 0 0) 0x7f8100085a50 con 0x7f811015d090 2023-06-23T00:35:52.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.338+0000 7f8114bd6700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f80f0066e00 0x7f80f00692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:52.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.338+0000 7f8114bd6700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f80f0066e00 0x7f80f00692b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f811015c870 tx=0x7f810800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:52.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.342+0000 7f810cff9700 1 -- 172.21.15.55:0/2868664548 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f8100053260 con 0x7f811015d090 2023-06-23T00:35:52.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:52.537+0000 7f8115bd8700 1 -- 172.21.15.55:0/2868664548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f811015dc10 con 0x7f811015d090 2023-06-23T00:35:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.207+0000 7f810cff9700 1 -- 172.21.15.55:0/2868664548 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f81000150c0 con 0x7f811015d090 2023-06-23T00:35:53.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 -- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f80f0066e00 msgr2=0x7f80f00692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f80f0066e00 0x7f80f00692b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f811015c870 tx=0x7f810800b040 comp rx=0 tx=0).stop 2023-06-23T00:35:53.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 -- 172.21.15.55:0/2868664548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f811015d090 msgr2=0x7f81100acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f811015d090 0x7f81100acfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f81100cd450 tx=0x7f8100017000 comp rx=0 tx=0).stop 2023-06-23T00:35:53.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 -- 172.21.15.55:0/2868664548 shutdown_connections 2023-06-23T00:35:53.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f80f0066e00 0x7f80f00692b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f811015d090 0x7f81100acfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f8110067410 0x7f81100ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 --2- 172.21.15.55:0/2868664548 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8110068190 0x7f81100aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 -- 172.21.15.55:0/2868664548 >> 172.21.15.55:0/2868664548 conn(0x7f81100cc8a0 msgr2=0x7f81101532e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:53.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 -- 172.21.15.55:0/2868664548 shutdown_connections 2023-06-23T00:35:53.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.210+0000 7f8115bd8700 1 -- 172.21.15.55:0/2868664548 wait complete. 2023-06-23T00:35:53.218 INFO:teuthology.orchestra.run.smithi055.stderr:enabled application 'rbd' on pool 'rbd' 2023-06-23T00:35:53.229 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-06-23T00:35:53.230 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-23T00:35:53.230 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-23T00:35:53.232 INFO:tasks.ceph.mds.a:Started 2023-06-23T00:35:53.233 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-23T00:35:53.234 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-23T00:35:53.236 INFO:tasks.ceph.mds.c:Started 2023-06-23T00:35:53.237 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-23T00:35:53.237 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-23T00:35:53.239 INFO:tasks.ceph.mds.b:Started 2023-06-23T00:35:53.240 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-23T00:35:53.240 DEBUG:teuthology.orchestra.run.smithi141:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-23T00:35:53.243 INFO:tasks.ceph.mds.d:Started 2023-06-23T00:35:53.243 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-06-23T00:35:53.244 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T00:35:53.316 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-06-23T00:35:53.317 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-06-23T00:35:53.322 INFO:tasks.ceph.mds.b.smithi141.stdout:starting mds.b at 2023-06-23T00:35:53.322 INFO:tasks.ceph.mds.d.smithi141.stdout:starting mds.d at 2023-06-23T00:35:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.370+0000 7fb02d817700 1 Processor -- start 2023-06-23T00:35:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.370+0000 7fb02d817700 1 -- start start 2023-06-23T00:35:53.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d817700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb028142de0 0x7fb0280b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d817700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb0280b54c0 0x7fb0280cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d817700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0280b2780 con 0x7fb0280b2310 2023-06-23T00:35:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d817700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0280cec20 con 0x7fb0280b54c0 2023-06-23T00:35:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d817700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280b5ae0 con 0x7fb028142de0 2023-06-23T00:35:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d016700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb0280b54c0 0x7fb0280cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb027fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb0280b2310 0x7fb0280b4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56890/0 (socket says 172.21.15.55:56890) 2023-06-23T00:35:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb027fff700 1 -- 172.21.15.55:0/1258242272 learned_addr learned my addr 172.21.15.55:0/1258242272 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02c815700 1 --2- 172.21.15.55:0/1258242272 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb028142de0 0x7fb0280b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d016700 1 -- 172.21.15.55:0/1258242272 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb028142de0 msgr2=0x7fb0280b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d016700 1 --2- 172.21.15.55:0/1258242272 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb028142de0 0x7fb0280b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d016700 1 -- 172.21.15.55:0/1258242272 >> v1:172.21.15.55:6789/0 conn(0x7fb0280b2310 legacy=0x7fb0280b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d016700 1 -- 172.21.15.55:0/1258242272 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0280b5dc0 con 0x7fb0280b54c0 2023-06-23T00:35:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.371+0000 7fb02d016700 1 --2- 172.21.15.55:0/1258242272 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b54c0 0x7fb0280cf0b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb020009e60 tx=0x7fb020009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebd33899413a6cc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb0277fe700 1 -- 172.21.15.55:0/1258242272 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02000e070 con 0x7fb0280b54c0 2023-06-23T00:35:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb0277fe700 1 -- 172.21.15.55:0/1258242272 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02000bc30 con 0x7fb0280b54c0 2023-06-23T00:35:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb0277fe700 1 -- 172.21.15.55:0/1258242272 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020012430 con 0x7fb0280b54c0 2023-06-23T00:35:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb02d817700 1 -- 172.21.15.55:0/1258242272 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b54c0 msgr2=0x7fb0280cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb02d817700 1 --2- 172.21.15.55:0/1258242272 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b54c0 0x7fb0280cf0b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb020009e60 tx=0x7fb020009780 comp rx=0 tx=0).stop 2023-06-23T00:35:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb02d817700 1 -- 172.21.15.55:0/1258242272 shutdown_connections 2023-06-23T00:35:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb02d817700 1 --2- 172.21.15.55:0/1258242272 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b54c0 0x7fb0280cf0b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb02d817700 1 --2- 172.21.15.55:0/1258242272 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb028142de0 0x7fb0280b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb02d817700 1 -- 172.21.15.55:0/1258242272 >> 172.21.15.55:0/1258242272 conn(0x7fb028070290 msgr2=0x7fb028070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb02d817700 1 -- 172.21.15.55:0/1258242272 shutdown_connections 2023-06-23T00:35:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.372+0000 7fb02d817700 1 -- 172.21.15.55:0/1258242272 wait complete. 2023-06-23T00:35:53.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.373+0000 7fb02d817700 1 Processor -- start 2023-06-23T00:35:53.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.373+0000 7fb02d817700 1 -- start start 2023-06-23T00:35:53.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02d817700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b2310 0x7fb028156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02d817700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb0280b54c0 0x7fb028140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02d817700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb028142de0 0x7fb028140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02d817700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02813bdd0 con 0x7fb0280b2310 2023-06-23T00:35:53.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02d817700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02815cc40 con 0x7fb028142de0 2023-06-23T00:35:53.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02d817700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02815cf20 con 0x7fb0280b54c0 2023-06-23T00:35:53.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02c815700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b2310 0x7fb028156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:53.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02c815700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b2310 0x7fb028156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39414/0 (socket says 172.21.15.55:39414) 2023-06-23T00:35:53.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02c815700 1 -- 172.21.15.55:0/2840427488 learned_addr learned my addr 172.21.15.55:0/2840427488 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:53.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb027fff700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb0280b54c0 0x7fb028140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:53.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02d016700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb028142de0 0x7fb028140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:53.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02c815700 1 -- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb0280b54c0 msgr2=0x7fb028140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02c815700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb0280b54c0 0x7fb028140750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.374+0000 7fb02c815700 1 -- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb028142de0 msgr2=0x7fb028140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.375+0000 7fb02c815700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb028142de0 0x7fb028140c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.375+0000 7fb02c815700 1 -- 172.21.15.55:0/2840427488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb020009a20 con 0x7fb0280b2310 2023-06-23T00:35:53.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.375+0000 7fb02c815700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b2310 0x7fb028156bc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00d860 tx=0x7fb01c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:53.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.375+0000 7fb024ff9700 1 -- 172.21.15.55:0/2840427488 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c009610 con 0x7fb0280b2310 2023-06-23T00:35:53.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.375+0000 7fb024ff9700 1 -- 172.21.15.55:0/2840427488 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c009770 con 0x7fb0280b2310 2023-06-23T00:35:53.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.375+0000 7fb02d817700 1 -- 172.21.15.55:0/2840427488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02815d200 con 0x7fb0280b2310 2023-06-23T00:35:53.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.375+0000 7fb024ff9700 1 -- 172.21.15.55:0/2840427488 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c009b30 con 0x7fb0280b2310 2023-06-23T00:35:53.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.375+0000 7fb02d817700 1 -- 172.21.15.55:0/2840427488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb028157100 con 0x7fb0280b2310 2023-06-23T00:35:53.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.376+0000 7fb02d817700 1 -- 172.21.15.55:0/2840427488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02813c100 con 0x7fb0280b2310 2023-06-23T00:35:53.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.376+0000 7fb024ff9700 1 -- 172.21.15.55:0/2840427488 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7fb01c01e460 con 0x7fb0280b2310 2023-06-23T00:35:53.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.377+0000 7fb024ff9700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb00c066e00 0x7fb00c0692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.377+0000 7fb024ff9700 1 -- 172.21.15.55:0/2840427488 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fb01c084800 con 0x7fb0280b2310 2023-06-23T00:35:53.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.377+0000 7fb027fff700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb00c066e00 0x7fb00c0692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:53.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.377+0000 7fb027fff700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb00c066e00 0x7fb00c0692b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0140058d0 tx=0x7fb014009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:53.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.381+0000 7fb024ff9700 1 -- 172.21.15.55:0/2840427488 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fb01c051d70 con 0x7fb0280b2310 2023-06-23T00:35:53.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.577+0000 7fb02d817700 1 -- 172.21.15.55:0/2840427488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb02815d720 con 0x7fb0280b2310 2023-06-23T00:35:53.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.578+0000 7fb024ff9700 1 -- 172.21.15.55:0/2840427488 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+13666 (secure 0 0 0) 0x7fb02815d720 con 0x7fb0280b2310 2023-06-23T00:35:53.582 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:53.583 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":15,"fsid":"c0a83a0b-dd71-486d-bdc4-a203f7ef404f","created":"2023-06-23T00:35:40.890983+0000","modified":"2023-06-23T00:35:53.194126+0000","last_up_change":"2023-06-23T00:35:51.173985+0000","last_in_change":"2023-06-23T00:35:43.351694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T00:35:52.176829+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T00:35:52.605332+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"d4590eaf-cc24-4ff1-8f1b-9637138bd300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":87956},{"type":"v1","addr":"172.21.15.55:6817","nonce":87956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":87956},{"type":"v1","addr":"172.21.15.55:6819","nonce":87956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":87956},{"type":"v1","addr":"172.21.15.55:6823","nonce":87956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":87956},{"type":"v1","addr":"172.21.15.55:6821","nonce":87956}]},"public_addr":"172.21.15.55:6817/87956","cluster_addr":"172.21.15.55:6819/87956","heartbeat_back_addr":"172.21.15.55:6823/87956","heartbeat_front_addr":"172.21.15.55:6821/87956","state":["exists","up"]},{"osd":1,"uuid":"518b1bc8-fb1c-405f-8346-fda7978829a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":87959},{"type":"v1","addr":"172.21.15.55:6809","nonce":87959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":87959},{"type":"v1","addr":"172.21.15.55:6811","nonce":87959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":87959},{"type":"v1","addr":"172.21.15.55:6815","nonce":87959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":87959},{"type":"v1","addr":"172.21.15.55:6813","nonce":87959}]},"public_addr":"172.21.15.55:6809/87959","cluster_addr":"172.21.15.55:6811/87959","heartbeat_back_addr":"172.21.15.55:6815/87959","heartbeat_front_addr":"172.21.15.55:6813/87959","state":["exists","up"]},{"osd":2,"uuid":"636a04f1-2379-43ca-9f02-0ee33d2af90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":87955},{"type":"v1","addr":"172.21.15.55:6801","nonce":87955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":87955},{"type":"v1","addr":"172.21.15.55:6803","nonce":87955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":87955},{"type":"v1","addr":"172.21.15.55:6807","nonce":87955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":87955},{"type":"v1","addr":"172.21.15.55:6805","nonce":87955}]},"public_addr":"172.21.15.55:6801/87955","cluster_addr":"172.21.15.55:6803/87955","heartbeat_back_addr":"172.21.15.55:6807/87955","heartbeat_front_addr":"172.21.15.55:6805/87955","state":["exists","up"]},{"osd":3,"uuid":"b893aff1-9f4d-42ca-9af6-42fd6ba3d92c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":87972},{"type":"v1","addr":"172.21.15.55:6825","nonce":87972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":87972},{"type":"v1","addr":"172.21.15.55:6827","nonce":87972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":87972},{"type":"v1","addr":"172.21.15.55:6831","nonce":87972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":87972},{"type":"v1","addr":"172.21.15.55:6829","nonce":87972}]},"public_addr":"172.21.15.55:6825/87972","cluster_addr":"172.21.15.55:6827/87972","heartbeat_back_addr":"172.21.15.55:6831/87972","heartbeat_front_addr":"172.21.15.55:6829/87972","state":["exists","up"]},{"osd":4,"uuid":"259d9777-c081-4c0b-9d83-46d348872e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":88177},{"type":"v1","addr":"172.21.15.141:6817","nonce":88177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":88177},{"type":"v1","addr":"172.21.15.141:6819","nonce":88177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6825","nonce":88177},{"type":"v1","addr":"172.21.15.141:6827","nonce":88177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6821","nonce":88177},{"type":"v1","addr":"172.21.15.141:6823","nonce":88177}]},"public_addr":"172.21.15.141:6817/88177","cluster_addr":"172.21.15.141:6819/88177","heartbeat_back_addr":"172.21.15.141:6827/88177","heartbeat_front_addr":"172.21.15.141:6823/88177","state":["exists","up"]},{"osd":5,"uuid":"56cfcc58-3f4d-4637-ae5d-a2a52dbdc37f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":88161},{"type":"v1","addr":"172.21.15.141:6801","nonce":88161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":88161},{"type":"v1","addr":"172.21.15.141:6803","nonce":88161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":88161},{"type":"v1","addr":"172.21.15.141:6807","nonce":88161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":88161},{"type":"v1","addr":"172.21.15.141:6805","nonce":88161}]},"public_addr":"172.21.15.141:6801/88161","cluster_addr":"172.21.15.141:6803/88161","heartbeat_back_addr":"172.21.15.141:6807/88161","heartbeat_front_addr":"172.21.15.141:6805/88161","state":["exists","up"]},{"osd":6,"uuid":"49f9b91b-16c2-4394-928e-97446f5feea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":88192},{"type":"v1","addr":"172.21.15.141:6822","nonce":88192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":88192},{"type":"v1","addr":"172.21.15.141:6826","nonce":88192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":88192},{"type":"v1","addr":"172.21.15.141:6831","nonce":88192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":88192},{"type":"v1","addr":"172.21.15.141:6829","nonce":88192}]},"public_addr":"172.21.15.141:6822/88192","cluster_addr":"172.21.15.141:6826/88192","heartbeat_back_addr":"172.21.15.141:6831/88192","heartbeat_front_addr":"172.21.15.141:6829/88192","state":["exists","up"]},{"osd":7,"uuid":"e9806e67-d1e0-49c5-a3e4-898bd4f840e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":88162},{"type":"v1","addr":"172.21.15.141:6809","nonce":88162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":88162},{"type":"v1","addr":"172.21.15.141:6811","nonce":88162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":88162},{"type":"v1","addr":"172.21.15.141:6815","nonce":88162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":88162},{"type":"v1","addr":"172.21.15.141:6813","nonce":88162}]},"public_addr":"172.21.15.141:6809/88162","cluster_addr":"172.21.15.141:6811/88162","heartbeat_back_addr":"172.21.15.141:6815/88162","heartbeat_front_addr":"172.21.15.141:6813/88162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.504877+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.470542+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.458869+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.499431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.550397+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.286978+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.541488+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.226174+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T00:35:53.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 -- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb00c066e00 msgr2=0x7fb00c0692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb00c066e00 0x7fb00c0692b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0140058d0 tx=0x7fb014009040 comp rx=0 tx=0).stop 2023-06-23T00:35:53.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 -- 172.21.15.55:0/2840427488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b2310 msgr2=0x7fb028156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b2310 0x7fb028156bc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00d860 tx=0x7fb01c008040 comp rx=0 tx=0).stop 2023-06-23T00:35:53.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 -- 172.21.15.55:0/2840427488 shutdown_connections 2023-06-23T00:35:53.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb00c066e00 0x7fb00c0692b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb028142de0 0x7fb028140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0280b2310 0x7fb028156bc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 --2- 172.21.15.55:0/2840427488 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb0280b54c0 0x7fb028140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 -- 172.21.15.55:0/2840427488 >> 172.21.15.55:0/2840427488 conn(0x7fb028070290 msgr2=0x7fb0280b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:53.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 -- 172.21.15.55:0/2840427488 shutdown_connections 2023-06-23T00:35:53.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.580+0000 7fb02d817700 1 -- 172.21.15.55:0/2840427488 wait complete. 2023-06-23T00:35:53.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T00:35:52.176829+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T00:35:52.605332+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}}] 2023-06-23T00:35:53.597 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T00:35:53.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.715+0000 7f7788c1b700 1 Processor -- start 2023-06-23T00:35:53.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.715+0000 7f7788c1b700 1 -- start start 2023-06-23T00:35:53.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.716+0000 7f7788c1b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f77840b3e00 0x7f77840b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.716+0000 7f7788c1b700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f77840b41d0 0x7f77840ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.716+0000 7f7788c1b700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77840ce0b0 con 0x7f7784067ff0 2023-06-23T00:35:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.716+0000 7f7788c1b700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77840cd4a0 con 0x7f77840b3e00 2023-06-23T00:35:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.716+0000 7f7788c1b700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77840cd780 con 0x7f77840b41d0 2023-06-23T00:35:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.716+0000 7f778259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f77840b3e00 0x7f77840b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.716+0000 7f7782d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7784067ff0 0x7f77840cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56894/0 (socket says 172.21.15.55:56894) 2023-06-23T00:35:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.716+0000 7f7782d9d700 1 -- 172.21.15.55:0/4279730012 learned_addr learned my addr 172.21.15.55:0/4279730012 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.716+0000 7f7781d9b700 1 --2- 172.21.15.55:0/4279730012 >> v2:172.21.15.141:3300/0 conn(0x7f77840b41d0 0x7f77840ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.716+0000 7f778259c700 1 -- 172.21.15.55:0/4279730012 >> v2:172.21.15.141:3300/0 conn(0x7f77840b41d0 msgr2=0x7f77840ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f778259c700 1 --2- 172.21.15.55:0/4279730012 >> v2:172.21.15.141:3300/0 conn(0x7f77840b41d0 0x7f77840ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f778259c700 1 -- 172.21.15.55:0/4279730012 >> v1:172.21.15.55:6789/0 conn(0x7f7784067ff0 legacy=0x7f77840cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f778259c700 1 -- 172.21.15.55:0/4279730012 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77840cda60 con 0x7f77840b3e00 2023-06-23T00:35:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f778259c700 1 --2- 172.21.15.55:0/4279730012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77840b3e00 0x7f77840b0760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f778409c330 tx=0x7f777c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8ba50b49d3ff67 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f778159a700 1 -- 172.21.15.55:0/4279730012 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c00e070 con 0x7f77840b3e00 2023-06-23T00:35:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f778159a700 1 -- 172.21.15.55:0/4279730012 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c00bbf0 con 0x7f77840b3e00 2023-06-23T00:35:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f778159a700 1 -- 172.21.15.55:0/4279730012 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c012430 con 0x7f77840b3e00 2023-06-23T00:35:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f7788c1b700 1 -- 172.21.15.55:0/4279730012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77840b3e00 msgr2=0x7f77840b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f7788c1b700 1 --2- 172.21.15.55:0/4279730012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77840b3e00 0x7f77840b0760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f778409c330 tx=0x7f777c009780 comp rx=0 tx=0).stop 2023-06-23T00:35:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f7788c1b700 1 -- 172.21.15.55:0/4279730012 shutdown_connections 2023-06-23T00:35:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f7788c1b700 1 --2- 172.21.15.55:0/4279730012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77840b3e00 0x7f77840b0760 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f7788c1b700 1 --2- 172.21.15.55:0/4279730012 >> v2:172.21.15.141:3300/0 conn(0x7f77840b41d0 0x7f77840ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f7788c1b700 1 -- 172.21.15.55:0/4279730012 >> 172.21.15.55:0/4279730012 conn(0x7f7784067bf0 msgr2=0x7f77840ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f7788c1b700 1 -- 172.21.15.55:0/4279730012 shutdown_connections 2023-06-23T00:35:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.717+0000 7f7788c1b700 1 -- 172.21.15.55:0/4279730012 wait complete. 2023-06-23T00:35:53.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.718+0000 7f7788c1b700 1 Processor -- start 2023-06-23T00:35:53.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.719+0000 7f7788c1b700 1 -- start start 2023-06-23T00:35:53.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.719+0000 7f7788c1b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7784067ff0 0x7f7784157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.719+0000 7f7788c1b700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f77840b3e00 0x7f778416aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.719+0000 7f7788c1b700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f77840b41d0 0x7f778416b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.719+0000 7f7788c1b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778409a9b0 con 0x7f7784067ff0 2023-06-23T00:35:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.719+0000 7f7788c1b700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77840ca890 con 0x7f77840b41d0 2023-06-23T00:35:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.719+0000 7f7788c1b700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778416be40 con 0x7f77840b3e00 2023-06-23T00:35:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.719+0000 7f778259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7784067ff0 0x7f7784157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.719+0000 7f778259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7784067ff0 0x7f7784157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39440/0 (socket says 172.21.15.55:39440) 2023-06-23T00:35:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.719+0000 7f778259c700 1 -- 172.21.15.55:0/721073832 learned_addr learned my addr 172.21.15.55:0/721073832 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f778259c700 1 -- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f77840b3e00 msgr2=0x7f778416aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f778259c700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f77840b3e00 0x7f778416aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f778259c700 1 -- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f77840b41d0 msgr2=0x7f778416b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f778259c700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f77840b41d0 0x7f778416b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f778259c700 1 -- 172.21.15.55:0/721073832 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c009a20 con 0x7f7784067ff0 2023-06-23T00:35:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f778259c700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7784067ff0 0x7f7784157310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f778406f920 tx=0x7f777c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f777a7fc700 1 -- 172.21.15.55:0/721073832 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c00beb0 con 0x7f7784067ff0 2023-06-23T00:35:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f777a7fc700 1 -- 172.21.15.55:0/721073832 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c021070 con 0x7f7784067ff0 2023-06-23T00:35:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f777a7fc700 1 -- 172.21.15.55:0/721073832 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0266a0 con 0x7f7784067ff0 2023-06-23T00:35:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f7788c1b700 1 -- 172.21.15.55:0/721073832 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778416bb60 con 0x7f7784067ff0 2023-06-23T00:35:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.720+0000 7f7788c1b700 1 -- 172.21.15.55:0/721073832 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778409a640 con 0x7f7784067ff0 2023-06-23T00:35:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.721+0000 7f7788c1b700 1 -- 172.21.15.55:0/721073832 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778404c1b0 con 0x7f7784067ff0 2023-06-23T00:35:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.721+0000 7f777a7fc700 1 -- 172.21.15.55:0/721073832 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7f777c023030 con 0x7f7784067ff0 2023-06-23T00:35:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.722+0000 7f777a7fc700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7764066e00 0x7f77640692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.722+0000 7f777a7fc700 1 -- 172.21.15.55:0/721073832 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f777c086270 con 0x7f7784067ff0 2023-06-23T00:35:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.722+0000 7f7781d9b700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7764066e00 0x7f77640692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:53.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.723+0000 7f7781d9b700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7764066e00 0x7f77640692b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f77840adac0 tx=0x7f776c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:53.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.726+0000 7f777a7fc700 1 -- 172.21.15.55:0/721073832 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f777c053880 con 0x7f7784067ff0 2023-06-23T00:35:53.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.920+0000 7f7788c1b700 1 -- 172.21.15.55:0/721073832 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f778406bd60 con 0x7f7784067ff0 2023-06-23T00:35:53.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.920+0000 7f777a7fc700 1 -- 172.21.15.55:0/721073832 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f777c056ea0 con 0x7f7784067ff0 2023-06-23T00:35:53.924 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-06-23T00:35:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 -- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7764066e00 msgr2=0x7f77640692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7764066e00 0x7f77640692b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f77840adac0 tx=0x7f776c0091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 -- 172.21.15.55:0/721073832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7784067ff0 msgr2=0x7f7784157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7784067ff0 0x7f7784157310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f778406f920 tx=0x7f777c019000 comp rx=0 tx=0).stop 2023-06-23T00:35:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 -- 172.21.15.55:0/721073832 shutdown_connections 2023-06-23T00:35:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7764066e00 0x7f77640692b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f77840b41d0 0x7f778416b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7784067ff0 0x7f7784157310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 --2- 172.21.15.55:0/721073832 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f77840b3e00 0x7f778416aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 -- 172.21.15.55:0/721073832 >> 172.21.15.55:0/721073832 conn(0x7f7784067bf0 msgr2=0x7f7784159170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 -- 172.21.15.55:0/721073832 shutdown_connections 2023-06-23T00:35:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:53.922+0000 7f7788c1b700 1 -- 172.21.15.55:0/721073832 wait complete. 2023-06-23T00:35:53.938 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T00:35:54.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.055+0000 7f180bb4c700 1 Processor -- start 2023-06-23T00:35:54.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.055+0000 7f180bb4c700 1 -- start start 2023-06-23T00:35:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.056+0000 7f180bb4c700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f18040b2720 0x7f18040d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.056+0000 7f180bb4c700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18040cedd0 con 0x7f1804153a80 2023-06-23T00:35:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.056+0000 7f180bb4c700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180415aed0 con 0x7f1804153e50 2023-06-23T00:35:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.056+0000 7f180bb4c700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18040d14c0 con 0x7f18040b2720 2023-06-23T00:35:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.056+0000 7f18098e8700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1804153a80 0x7f180415a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56896/0 (socket says 172.21.15.55:56896) 2023-06-23T00:35:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.056+0000 7f18098e8700 1 -- 172.21.15.55:0/535843072 learned_addr learned my addr 172.21.15.55:0/535843072 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:54.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.056+0000 7f180a0e9700 1 --2- 172.21.15.55:0/535843072 >> v2:172.21.15.141:3300/0 conn(0x7f18040b2720 0x7f18040d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.056+0000 7f18088e6700 1 -- 172.21.15.55:0/535843072 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811502637 0 0) 0x7f18040cedd0 con 0x7f1804153a80 2023-06-23T00:35:54.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.056+0000 7f18088e6700 1 -- 172.21.15.55:0/535843072 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17ec003570 con 0x7f1804153a80 2023-06-23T00:35:54.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f18088e6700 1 -- 172.21.15.55:0/535843072 <== mon.1 v1:172.21.15.141:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216429050 0 0) 0x7f180415aed0 con 0x7f1804153e50 2023-06-23T00:35:54.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f18088e6700 1 -- 172.21.15.55:0/535843072 --> v1:172.21.15.141:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18040cedd0 con 0x7f1804153e50 2023-06-23T00:35:54.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f180a0e9700 1 -- 172.21.15.55:0/535843072 >> v1:172.21.15.141:6789/0 conn(0x7f1804153e50 legacy=0x7f18040b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f180a0e9700 1 -- 172.21.15.55:0/535843072 >> v1:172.21.15.55:6789/0 conn(0x7f1804153a80 legacy=0x7f180415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f180a0e9700 1 -- 172.21.15.55:0/535843072 --> v2:172.21.15.141:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18040ceac0 con 0x7f18040b2720 2023-06-23T00:35:54.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f180a0e9700 1 --2- 172.21.15.55:0/535843072 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f18040b2720 0x7f18040d0590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009e60 tx=0x7f17f4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e9a4b9fcd4e23e3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:54.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f18088e6700 1 -- 172.21.15.55:0/535843072 <== mon.1 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f400e070 con 0x7f18040b2720 2023-06-23T00:35:54.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f18088e6700 1 -- 172.21.15.55:0/535843072 <== mon.1 v2:172.21.15.141:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f400bab0 con 0x7f18040b2720 2023-06-23T00:35:54.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f18088e6700 1 -- 172.21.15.55:0/535843072 <== mon.1 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f400be70 con 0x7f18040b2720 2023-06-23T00:35:54.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f180bb4c700 1 -- 172.21.15.55:0/535843072 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f18040b2720 msgr2=0x7f18040d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.057+0000 7f180bb4c700 1 --2- 172.21.15.55:0/535843072 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f18040b2720 0x7f18040d0590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009e60 tx=0x7f17f4009780 comp rx=0 tx=0).stop 2023-06-23T00:35:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.058+0000 7f180bb4c700 1 -- 172.21.15.55:0/535843072 shutdown_connections 2023-06-23T00:35:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.058+0000 7f180bb4c700 1 --2- 172.21.15.55:0/535843072 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f18040b2720 0x7f18040d0590 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.058+0000 7f180bb4c700 1 -- 172.21.15.55:0/535843072 >> 172.21.15.55:0/535843072 conn(0x7f1804070290 msgr2=0x7f1804070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.058+0000 7f180bb4c700 1 -- 172.21.15.55:0/535843072 shutdown_connections 2023-06-23T00:35:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.058+0000 7f180bb4c700 1 -- 172.21.15.55:0/535843072 wait complete. 2023-06-23T00:35:54.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.059+0000 7f180bb4c700 1 Processor -- start 2023-06-23T00:35:54.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.059+0000 7f180bb4c700 1 -- start start 2023-06-23T00:35:54.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.059+0000 7f180bb4c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18040b2720 0x7f1804159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.059+0000 7f180bb4c700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1804153a80 0x7f1804155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.059+0000 7f180bb4c700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1804153e50 0x7f1804150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.059+0000 7f180bb4c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1804155fe0 con 0x7f18040b2720 2023-06-23T00:35:54.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.059+0000 7f180bb4c700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18040b03a0 con 0x7f1804153e50 2023-06-23T00:35:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.059+0000 7f180bb4c700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18041510f0 con 0x7f1804153a80 2023-06-23T00:35:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f18098e8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18040b2720 0x7f1804159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f18098e8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18040b2720 0x7f1804159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39450/0 (socket says 172.21.15.55:39450) 2023-06-23T00:35:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f18098e8700 1 -- 172.21.15.55:0/3191293590 learned_addr learned my addr 172.21.15.55:0/3191293590 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f180a0e9700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1804153e50 0x7f1804150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f18090e7700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1804153a80 0x7f1804155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f18098e8700 1 -- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1804153a80 msgr2=0x7f1804155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f18098e8700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1804153a80 0x7f1804155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f18098e8700 1 -- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1804153e50 msgr2=0x7f1804150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f18098e8700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1804153e50 0x7f1804150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f18098e8700 1 -- 172.21.15.55:0/3191293590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f4009a20 con 0x7f18040b2720 2023-06-23T00:35:54.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f18098e8700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18040b2720 0x7f1804159180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00c790 tx=0x7f17fc007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:54.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f1801ffb700 1 -- 172.21.15.55:0/3191293590 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc005b30 con 0x7f18040b2720 2023-06-23T00:35:54.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f1801ffb700 1 -- 172.21.15.55:0/3191293590 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc005c90 con 0x7f18040b2720 2023-06-23T00:35:54.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f1801ffb700 1 -- 172.21.15.55:0/3191293590 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc007300 con 0x7f18040b2720 2023-06-23T00:35:54.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.060+0000 7f180bb4c700 1 -- 172.21.15.55:0/3191293590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1804155a20 con 0x7f18040b2720 2023-06-23T00:35:54.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.061+0000 7f180bb4c700 1 -- 172.21.15.55:0/3191293590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18040b0030 con 0x7f18040b2720 2023-06-23T00:35:54.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.062+0000 7f180bb4c700 1 -- 172.21.15.55:0/3191293590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180404c1b0 con 0x7f18040b2720 2023-06-23T00:35:54.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.062+0000 7f1801ffb700 1 -- 172.21.15.55:0/3191293590 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7f17fc01b3b0 con 0x7f18040b2720 2023-06-23T00:35:54.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.062+0000 7f1801ffb700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f17f0066e00 0x7f17f00692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.062+0000 7f1801ffb700 1 -- 172.21.15.55:0/3191293590 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f17fc083650 con 0x7f18040b2720 2023-06-23T00:35:54.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.062+0000 7f18090e7700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f17f0066e00 0x7f17f00692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.063+0000 7f18090e7700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f17f0066e00 0x7f17f00692b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f18041548c0 tx=0x7f17f8007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:54.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.066+0000 7f1801ffb700 1 -- 172.21.15.55:0/3191293590 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f17fc050c60 con 0x7f18040b2720 2023-06-23T00:35:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.266+0000 7f180bb4c700 1 -- 172.21.15.55:0/3191293590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18040d0c20 con 0x7f18040b2720 2023-06-23T00:35:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.267+0000 7f1801ffb700 1 -- 172.21.15.55:0/3191293590 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17fc017020 con 0x7f18040b2720 2023-06-23T00:35:54.271 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-06-23T00:35:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 -- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f17f0066e00 msgr2=0x7f17f00692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f17f0066e00 0x7f17f00692b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f18041548c0 tx=0x7f17f8007d50 comp rx=0 tx=0).stop 2023-06-23T00:35:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 -- 172.21.15.55:0/3191293590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18040b2720 msgr2=0x7f1804159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18040b2720 0x7f1804159180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00c790 tx=0x7f17fc007ea0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 -- 172.21.15.55:0/3191293590 shutdown_connections 2023-06-23T00:35:54.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f17f0066e00 0x7f17f00692b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1804153e50 0x7f1804150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18040b2720 0x7f1804159180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 --2- 172.21.15.55:0/3191293590 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1804153a80 0x7f1804155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 -- 172.21.15.55:0/3191293590 >> 172.21.15.55:0/3191293590 conn(0x7f1804070290 msgr2=0x7f180415e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:54.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 -- 172.21.15.55:0/3191293590 shutdown_connections 2023-06-23T00:35:54.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.269+0000 7f180bb4c700 1 -- 172.21.15.55:0/3191293590 wait complete. 2023-06-23T00:35:54.288 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T00:35:54.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.408+0000 7f993501d700 1 Processor -- start 2023-06-23T00:35:54.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.408+0000 7f993501d700 1 -- start start 2023-06-23T00:35:54.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.409+0000 7f993501d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f99300af4b0 0x7f993015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.409+0000 7f993501d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99300acc50 con 0x7f993015b400 2023-06-23T00:35:54.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.409+0000 7f993501d700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99300acf30 con 0x7f99301518e0 2023-06-23T00:35:54.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.409+0000 7f993501d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99300af930 con 0x7f99300af4b0 2023-06-23T00:35:54.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.409+0000 7f992e59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f99300af4b0 0x7f993015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.409+0000 7f992e59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f99300af4b0 0x7f993015b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39464/0 (socket says 172.21.15.55:39464) 2023-06-23T00:35:54.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.409+0000 7f992e59c700 1 -- 172.21.15.55:0/478103477 learned_addr learned my addr 172.21.15.55:0/478103477 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:54.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.409+0000 7f992ed9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f993015b400 0x7f99300ab900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56906/0 (socket says 172.21.15.55:56906) 2023-06-23T00:35:54.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f992e59c700 1 -- 172.21.15.55:0/478103477 >> v1:172.21.15.141:6789/0 conn(0x7f99301518e0 legacy=0x7f9930151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:54.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f992e59c700 1 -- 172.21.15.55:0/478103477 >> v1:172.21.15.55:6789/0 conn(0x7f993015b400 legacy=0x7f99300ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f992e59c700 1 -- 172.21.15.55:0/478103477 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993015bd10 con 0x7f99300af4b0 2023-06-23T00:35:54.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f992dd9b700 1 -- 172.21.15.55:0/478103477 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351031553 0 0) 0x7f99300acc50 con 0x7f993015b400 2023-06-23T00:35:54.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f992e59c700 1 --2- 172.21.15.55:0/478103477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300af4b0 0x7f993015b7d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99180099b0 tx=0x7f9918009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454e18427efe5869 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:54.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f992dd9b700 1 -- 172.21.15.55:0/478103477 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800b820 con 0x7f99300af4b0 2023-06-23T00:35:54.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f992dd9b700 1 -- 172.21.15.55:0/478103477 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991800b980 con 0x7f99300af4b0 2023-06-23T00:35:54.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f992dd9b700 1 -- 172.21.15.55:0/478103477 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800bd20 con 0x7f99300af4b0 2023-06-23T00:35:54.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f993501d700 1 -- 172.21.15.55:0/478103477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300af4b0 msgr2=0x7f993015b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f993501d700 1 --2- 172.21.15.55:0/478103477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300af4b0 0x7f993015b7d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99180099b0 tx=0x7f9918009250 comp rx=0 tx=0).stop 2023-06-23T00:35:54.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f993501d700 1 -- 172.21.15.55:0/478103477 shutdown_connections 2023-06-23T00:35:54.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f993501d700 1 --2- 172.21.15.55:0/478103477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300af4b0 0x7f993015b7d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f993501d700 1 -- 172.21.15.55:0/478103477 >> 172.21.15.55:0/478103477 conn(0x7f9930067bf0 msgr2=0x7f9930087f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:54.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f993501d700 1 -- 172.21.15.55:0/478103477 shutdown_connections 2023-06-23T00:35:54.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.410+0000 7f993501d700 1 -- 172.21.15.55:0/478103477 wait complete. 2023-06-23T00:35:54.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.412+0000 7f993501d700 1 Processor -- start 2023-06-23T00:35:54.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.412+0000 7f993501d700 1 -- start start 2023-06-23T00:35:54.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.412+0000 7f993501d700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f99300af4b0 0x7f99300ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.412+0000 7f993501d700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f99301518e0 0x7f99301696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.412+0000 7f993501d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f993015b400 0x7f9930169c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.412+0000 7f993501d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9930069d90 con 0x7f993015b400 2023-06-23T00:35:54.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.412+0000 7f993501d700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9930069180 con 0x7f99300af4b0 2023-06-23T00:35:54.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.412+0000 7f993501d700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9930069460 con 0x7f99301518e0 2023-06-23T00:35:54.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.412+0000 7f992f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f993015b400 0x7f9930169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f992f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f993015b400 0x7f9930169c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39472/0 (socket says 172.21.15.55:39472) 2023-06-23T00:35:54.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f992f59e700 1 -- 172.21.15.55:0/4205377440 learned_addr learned my addr 172.21.15.55:0/4205377440 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:54.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f992e59c700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f99301518e0 0x7f99301696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f992ed9d700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f99300af4b0 0x7f99300ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f992f59e700 1 -- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f99301518e0 msgr2=0x7f99301696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f992f59e700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f99301518e0 0x7f99301696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f992f59e700 1 -- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f99300af4b0 msgr2=0x7f99300ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f992f59e700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f99300af4b0 0x7f99300ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f992f59e700 1 -- 172.21.15.55:0/4205377440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180094f0 con 0x7f993015b400 2023-06-23T00:35:54.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f992f59e700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f993015b400 0x7f9930169c30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f992800c930 tx=0x7f99280026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:54.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f9926ffd700 1 -- 172.21.15.55:0/4205377440 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800ce70 con 0x7f993015b400 2023-06-23T00:35:54.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f9926ffd700 1 -- 172.21.15.55:0/4205377440 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992800a7e0 con 0x7f993015b400 2023-06-23T00:35:54.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f993501d700 1 -- 172.21.15.55:0/4205377440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9930069740 con 0x7f993015b400 2023-06-23T00:35:54.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f9926ffd700 1 -- 172.21.15.55:0/4205377440 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800aba0 con 0x7f993015b400 2023-06-23T00:35:54.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.413+0000 7f993501d700 1 -- 172.21.15.55:0/4205377440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9930068d70 con 0x7f993015b400 2023-06-23T00:35:54.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.414+0000 7f993501d700 1 -- 172.21.15.55:0/4205377440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9930069be0 con 0x7f993015b400 2023-06-23T00:35:54.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.415+0000 7f9926ffd700 1 -- 172.21.15.55:0/4205377440 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7f9928006ba0 con 0x7f993015b400 2023-06-23T00:35:54.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.415+0000 7f9926ffd700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f9910066e00 0x7f99100692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.415+0000 7f9926ffd700 1 -- 172.21.15.55:0/4205377440 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9928084a10 con 0x7f993015b400 2023-06-23T00:35:54.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.415+0000 7f992ed9d700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f9910066e00 0x7f99100692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.416+0000 7f992ed9d700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f9910066e00 0x7f99100692b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99200027b0 tx=0x7f9920009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:54.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.419+0000 7f9926ffd700 1 -- 172.21.15.55:0/4205377440 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f9928052140 con 0x7f993015b400 2023-06-23T00:35:54.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.669+0000 7f993501d700 1 -- 172.21.15.55:0/4205377440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99300b0d90 con 0x7f993015b400 2023-06-23T00:35:54.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.669+0000 7f9926ffd700 1 -- 172.21.15.55:0/4205377440 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f9928016090 con 0x7f993015b400 2023-06-23T00:35:54.674 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:54.674 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6836/3128426312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6834","nonce":3128426312},{"type":"v1","addr":"172.21.15.141:6836","nonce":3128426312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.141:6837/1655690097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6835","nonce":1655690097},{"type":"v1","addr":"172.21.15.141:6837","nonce":1655690097}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/2922232663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2922232663},{"type":"v1","addr":"172.21.15.55:6834","nonce":2922232663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2847237397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2847237397},{"type":"v1","addr":"172.21.15.55:6835","nonce":2847237397}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-06-23T00:35:54.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 -- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f9910066e00 msgr2=0x7f99100692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f9910066e00 0x7f99100692b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99200027b0 tx=0x7f9920009040 comp rx=0 tx=0).stop 2023-06-23T00:35:54.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 -- 172.21.15.55:0/4205377440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f993015b400 msgr2=0x7f9930169c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f993015b400 0x7f9930169c30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f992800c930 tx=0x7f99280026e0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 -- 172.21.15.55:0/4205377440 shutdown_connections 2023-06-23T00:35:54.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f9910066e00 0x7f99100692b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f993015b400 0x7f9930169c30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f99300af4b0 0x7f99300ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 --2- 172.21.15.55:0/4205377440 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f99301518e0 0x7f99301696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 -- 172.21.15.55:0/4205377440 >> 172.21.15.55:0/4205377440 conn(0x7f9930067bf0 msgr2=0x7f9930151cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 -- 172.21.15.55:0/4205377440 shutdown_connections 2023-06-23T00:35:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.672+0000 7f993501d700 1 -- 172.21.15.55:0/4205377440 wait complete. 2023-06-23T00:35:54.678 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 3 2023-06-23T00:35:54.688 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T00:35:54.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.806+0000 7fb74d196700 1 Processor -- start 2023-06-23T00:35:54.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.806+0000 7fb74d196700 1 -- start start 2023-06-23T00:35:54.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb74d196700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb748142de0 0x7fb7480b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb74d196700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7fb7480b54c0 0x7fb7480cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb74d196700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7480b2780 con 0x7fb7480b2310 2023-06-23T00:35:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb74d196700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7480b58e0 con 0x7fb748142de0 2023-06-23T00:35:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb74d196700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7480cec20 con 0x7fb7480b54c0 2023-06-23T00:35:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb747fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb748142de0 0x7fb7480b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb747fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb748142de0 0x7fb7480b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39478/0 (socket says 172.21.15.55:39478) 2023-06-23T00:35:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb747fff700 1 -- 172.21.15.55:0/1981893284 learned_addr learned my addr 172.21.15.55:0/1981893284 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb74c995700 1 --2- 172.21.15.55:0/1981893284 >> v2:172.21.15.141:3300/0 conn(0x7fb7480b54c0 0x7fb7480cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb747fff700 1 -- 172.21.15.55:0/1981893284 >> v2:172.21.15.141:3300/0 conn(0x7fb7480b54c0 msgr2=0x7fb7480cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb747fff700 1 --2- 172.21.15.55:0/1981893284 >> v2:172.21.15.141:3300/0 conn(0x7fb7480b54c0 0x7fb7480cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb747fff700 1 -- 172.21.15.55:0/1981893284 >> v1:172.21.15.141:6789/0 conn(0x7fb7480b2310 legacy=0x7fb7480b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb747fff700 1 -- 172.21.15.55:0/1981893284 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7480b5ae0 con 0x7fb748142de0 2023-06-23T00:35:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.807+0000 7fb747fff700 1 --2- 172.21.15.55:0/1981893284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 0x7fb7480b1dd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7480b3370 tx=0x7fb730009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471ebd5749c78fb2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb746ffd700 1 -- 172.21.15.55:0/1981893284 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730009e70 con 0x7fb748142de0 2023-06-23T00:35:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb746ffd700 1 -- 172.21.15.55:0/1981893284 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73000b860 con 0x7fb748142de0 2023-06-23T00:35:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb746ffd700 1 -- 172.21.15.55:0/1981893284 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73000bc00 con 0x7fb748142de0 2023-06-23T00:35:54.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb74d196700 1 -- 172.21.15.55:0/1981893284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 msgr2=0x7fb7480b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb74d196700 1 --2- 172.21.15.55:0/1981893284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 0x7fb7480b1dd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7480b3370 tx=0x7fb730009250 comp rx=0 tx=0).stop 2023-06-23T00:35:54.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb74d196700 1 -- 172.21.15.55:0/1981893284 shutdown_connections 2023-06-23T00:35:54.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb74d196700 1 --2- 172.21.15.55:0/1981893284 >> v2:172.21.15.141:3300/0 conn(0x7fb7480b54c0 0x7fb7480cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb74d196700 1 --2- 172.21.15.55:0/1981893284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 0x7fb7480b1dd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb74d196700 1 -- 172.21.15.55:0/1981893284 >> 172.21.15.55:0/1981893284 conn(0x7fb748070290 msgr2=0x7fb748070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:54.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb74d196700 1 -- 172.21.15.55:0/1981893284 shutdown_connections 2023-06-23T00:35:54.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.808+0000 7fb74d196700 1 -- 172.21.15.55:0/1981893284 wait complete. 2023-06-23T00:35:54.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.809+0000 7fb74d196700 1 Processor -- start 2023-06-23T00:35:54.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.809+0000 7fb74d196700 1 -- start start 2023-06-23T00:35:54.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb74d196700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb7480b2310 0x7fb748158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb74d196700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb7480b54c0 0x7fb748169a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb74d196700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 0x7fb74815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb74d196700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7481400f0 con 0x7fb748142de0 2023-06-23T00:35:54.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb74d196700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74813fb30 con 0x7fb7480b54c0 2023-06-23T00:35:54.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb74d196700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74813fe10 con 0x7fb7480b2310 2023-06-23T00:35:54.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb74c995700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 0x7fb74815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb74c995700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 0x7fb74815df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39492/0 (socket says 172.21.15.55:39492) 2023-06-23T00:35:54.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb74c995700 1 -- 172.21.15.55:0/3987746730 learned_addr learned my addr 172.21.15.55:0/3987746730 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:54.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb747fff700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb7480b2310 0x7fb748158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.810+0000 7fb7477fe700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb7480b54c0 0x7fb748169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb74c995700 1 -- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb7480b2310 msgr2=0x7fb748158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb74c995700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb7480b2310 0x7fb748158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb74c995700 1 -- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb7480b54c0 msgr2=0x7fb748169a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb74c995700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb7480b54c0 0x7fb748169a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb74c995700 1 -- 172.21.15.55:0/3987746730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300094f0 con 0x7fb748142de0 2023-06-23T00:35:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb74c995700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 0x7fb74815df00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb740009e90 tx=0x7fb74000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb73ffff700 1 -- 172.21.15.55:0/3987746730 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74000cb70 con 0x7fb748142de0 2023-06-23T00:35:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb73ffff700 1 -- 172.21.15.55:0/3987746730 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74000f040 con 0x7fb748142de0 2023-06-23T00:35:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb74d196700 1 -- 172.21.15.55:0/3987746730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74813d6c0 con 0x7fb748142de0 2023-06-23T00:35:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb73ffff700 1 -- 172.21.15.55:0/3987746730 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740014480 con 0x7fb748142de0 2023-06-23T00:35:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.811+0000 7fb74d196700 1 -- 172.21.15.55:0/3987746730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb748169f90 con 0x7fb748142de0 2023-06-23T00:35:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.812+0000 7fb74d196700 1 -- 172.21.15.55:0/3987746730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74813db60 con 0x7fb748142de0 2023-06-23T00:35:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.812+0000 7fb73ffff700 1 -- 172.21.15.55:0/3987746730 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7fb740014830 con 0x7fb748142de0 2023-06-23T00:35:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.813+0000 7fb73ffff700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb728066e00 0x7fb7280692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.813+0000 7fb73ffff700 1 -- 172.21.15.55:0/3987746730 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb740084d90 con 0x7fb748142de0 2023-06-23T00:35:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.813+0000 7fb747fff700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb728066e00 0x7fb7280692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.813+0000 7fb747fff700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb728066e00 0x7fb7280692b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb7480b3370 tx=0x7fb730017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:54.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:54.817+0000 7fb73ffff700 1 -- 172.21.15.55:0/3987746730 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fb740088030 con 0x7fb748142de0 2023-06-23T00:35:55.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.013+0000 7fb74d196700 1 -- 172.21.15.55:0/3987746730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb74813d510 con 0x7fb748142de0 2023-06-23T00:35:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.014+0000 7fb73ffff700 1 -- 172.21.15.55:0/3987746730 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13661 (secure 0 0 0) 0x7fb740052390 con 0x7fb748142de0 2023-06-23T00:35:55.019 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:55.020 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"fsid":"c0a83a0b-dd71-486d-bdc4-a203f7ef404f","created":"2023-06-23T00:35:40.890983+0000","modified":"2023-06-23T00:35:54.200583+0000","last_up_change":"2023-06-23T00:35:51.173985+0000","last_in_change":"2023-06-23T00:35:43.351694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T00:35:52.176829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T00:35:52.605332+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d4590eaf-cc24-4ff1-8f1b-9637138bd300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":87956},{"type":"v1","addr":"172.21.15.55:6817","nonce":87956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":87956},{"type":"v1","addr":"172.21.15.55:6819","nonce":87956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":87956},{"type":"v1","addr":"172.21.15.55:6823","nonce":87956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":87956},{"type":"v1","addr":"172.21.15.55:6821","nonce":87956}]},"public_addr":"172.21.15.55:6817/87956","cluster_addr":"172.21.15.55:6819/87956","heartbeat_back_addr":"172.21.15.55:6823/87956","heartbeat_front_addr":"172.21.15.55:6821/87956","state":["exists","up"]},{"osd":1,"uuid":"518b1bc8-fb1c-405f-8346-fda7978829a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":87959},{"type":"v1","addr":"172.21.15.55:6809","nonce":87959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":87959},{"type":"v1","addr":"172.21.15.55:6811","nonce":87959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":87959},{"type":"v1","addr":"172.21.15.55:6815","nonce":87959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":87959},{"type":"v1","addr":"172.21.15.55:6813","nonce":87959}]},"public_addr":"172.21.15.55:6809/87959","cluster_addr":"172.21.15.55:6811/87959","heartbeat_back_addr":"172.21.15.55:6815/87959","heartbeat_front_addr":"172.21.15.55:6813/87959","state":["exists","up"]},{"osd":2,"uuid":"636a04f1-2379-43ca-9f02-0ee33d2af90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":87955},{"type":"v1","addr":"172.21.15.55:6801","nonce":87955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":87955},{"type":"v1","addr":"172.21.15.55:6803","nonce":87955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":87955},{"type":"v1","addr":"172.21.15.55:6807","nonce":87955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":87955},{"type":"v1","addr":"172.21.15.55:6805","nonce":87955}]},"public_addr":"172.21.15.55:6801/87955","cluster_addr":"172.21.15.55:6803/87955","heartbeat_back_addr":"172.21.15.55:6807/87955","heartbeat_front_addr":"172.21.15.55:6805/87955","state":["exists","up"]},{"osd":3,"uuid":"b893aff1-9f4d-42ca-9af6-42fd6ba3d92c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":87972},{"type":"v1","addr":"172.21.15.55:6825","nonce":87972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":87972},{"type":"v1","addr":"172.21.15.55:6827","nonce":87972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":87972},{"type":"v1","addr":"172.21.15.55:6831","nonce":87972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":87972},{"type":"v1","addr":"172.21.15.55:6829","nonce":87972}]},"public_addr":"172.21.15.55:6825/87972","cluster_addr":"172.21.15.55:6827/87972","heartbeat_back_addr":"172.21.15.55:6831/87972","heartbeat_front_addr":"172.21.15.55:6829/87972","state":["exists","up"]},{"osd":4,"uuid":"259d9777-c081-4c0b-9d83-46d348872e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":88177},{"type":"v1","addr":"172.21.15.141:6817","nonce":88177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":88177},{"type":"v1","addr":"172.21.15.141:6819","nonce":88177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6825","nonce":88177},{"type":"v1","addr":"172.21.15.141:6827","nonce":88177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6821","nonce":88177},{"type":"v1","addr":"172.21.15.141:6823","nonce":88177}]},"public_addr":"172.21.15.141:6817/88177","cluster_addr":"172.21.15.141:6819/88177","heartbeat_back_addr":"172.21.15.141:6827/88177","heartbeat_front_addr":"172.21.15.141:6823/88177","state":["exists","up"]},{"osd":5,"uuid":"56cfcc58-3f4d-4637-ae5d-a2a52dbdc37f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":88161},{"type":"v1","addr":"172.21.15.141:6801","nonce":88161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":88161},{"type":"v1","addr":"172.21.15.141:6803","nonce":88161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":88161},{"type":"v1","addr":"172.21.15.141:6807","nonce":88161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":88161},{"type":"v1","addr":"172.21.15.141:6805","nonce":88161}]},"public_addr":"172.21.15.141:6801/88161","cluster_addr":"172.21.15.141:6803/88161","heartbeat_back_addr":"172.21.15.141:6807/88161","heartbeat_front_addr":"172.21.15.141:6805/88161","state":["exists","up"]},{"osd":6,"uuid":"49f9b91b-16c2-4394-928e-97446f5feea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":88192},{"type":"v1","addr":"172.21.15.141:6822","nonce":88192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":88192},{"type":"v1","addr":"172.21.15.141:6826","nonce":88192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":88192},{"type":"v1","addr":"172.21.15.141:6831","nonce":88192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":88192},{"type":"v1","addr":"172.21.15.141:6829","nonce":88192}]},"public_addr":"172.21.15.141:6822/88192","cluster_addr":"172.21.15.141:6826/88192","heartbeat_back_addr":"172.21.15.141:6831/88192","heartbeat_front_addr":"172.21.15.141:6829/88192","state":["exists","up"]},{"osd":7,"uuid":"e9806e67-d1e0-49c5-a3e4-898bd4f840e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":88162},{"type":"v1","addr":"172.21.15.141:6809","nonce":88162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":88162},{"type":"v1","addr":"172.21.15.141:6811","nonce":88162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":88162},{"type":"v1","addr":"172.21.15.141:6815","nonce":88162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":88162},{"type":"v1","addr":"172.21.15.141:6813","nonce":88162}]},"public_addr":"172.21.15.141:6809/88162","cluster_addr":"172.21.15.141:6811/88162","heartbeat_back_addr":"172.21.15.141:6815/88162","heartbeat_front_addr":"172.21.15.141:6813/88162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.504877+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.470542+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.458869+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.499431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.550397+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.286978+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.541488+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.226174+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T00:35:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 -- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb728066e00 msgr2=0x7fb7280692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb728066e00 0x7fb7280692b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb7480b3370 tx=0x7fb730017000 comp rx=0 tx=0).stop 2023-06-23T00:35:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 -- 172.21.15.55:0/3987746730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 msgr2=0x7fb74815df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 0x7fb74815df00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb740009e90 tx=0x7fb74000b040 comp rx=0 tx=0).stop 2023-06-23T00:35:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 -- 172.21.15.55:0/3987746730 shutdown_connections 2023-06-23T00:35:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fb728066e00 0x7fb7280692b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb748142de0 0x7fb74815df00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fb7480b2310 0x7fb748158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 --2- 172.21.15.55:0/3987746730 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fb7480b54c0 0x7fb748169a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 -- 172.21.15.55:0/3987746730 >> 172.21.15.55:0/3987746730 conn(0x7fb748070290 msgr2=0x7fb7480cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:55.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 -- 172.21.15.55:0/3987746730 shutdown_connections 2023-06-23T00:35:55.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.016+0000 7fb74d196700 1 -- 172.21.15.55:0/3987746730 wait complete. 2023-06-23T00:35:55.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-23T00:35:52.176829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-23T00:35:52.605332+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-06-23T00:35:55.033 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T00:35:55.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.153+0000 7efed11aa700 1 Processor -- start 2023-06-23T00:35:55.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.153+0000 7efed11aa700 1 -- start start 2023-06-23T00:35:55.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.154+0000 7efed11aa700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc06a670 0x7efecc161340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.154+0000 7efed11aa700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc0c7a20 con 0x7efecc159a40 2023-06-23T00:35:55.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.154+0000 7efed11aa700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc0c9e50 con 0x7efecc159e10 2023-06-23T00:35:55.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.154+0000 7efed11aa700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0b0c10 con 0x7efecc06a670 2023-06-23T00:35:55.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.154+0000 7efecad9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efecc159a40 0x7efecc0b0110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56922/0 (socket says 172.21.15.55:56922) 2023-06-23T00:35:55.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.154+0000 7efecad9d700 1 -- 172.21.15.55:0/3144552198 learned_addr learned my addr 172.21.15.55:0/3144552198 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:55.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.154+0000 7efecb59e700 1 --2- 172.21.15.55:0/3144552198 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc06a670 0x7efecc161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.154+0000 7efec9d9b700 1 -- 172.21.15.55:0/3144552198 <== mon.1 v1:172.21.15.141:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043952174 0 0) 0x7efecc0c9e50 con 0x7efecc159e10 2023-06-23T00:35:55.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.154+0000 7efecb59e700 1 -- 172.21.15.55:0/3144552198 >> v1:172.21.15.141:6789/0 conn(0x7efecc159e10 legacy=0x7efecc069f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.155+0000 7efecb59e700 1 -- 172.21.15.55:0/3144552198 >> v1:172.21.15.55:6789/0 conn(0x7efecc159a40 legacy=0x7efecc0b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.155+0000 7efecb59e700 1 -- 172.21.15.55:0/3144552198 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0b0ef0 con 0x7efecc06a670 2023-06-23T00:35:55.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.155+0000 7efecb59e700 1 --2- 172.21.15.55:0/3144552198 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc06a670 0x7efecc161340 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efec4009e90 tx=0x7efec4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=772dd89d691af885 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:55.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.155+0000 7efec9d9b700 1 -- 172.21.15.55:0/3144552198 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec400bc30 con 0x7efecc06a670 2023-06-23T00:35:55.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.155+0000 7efec9d9b700 1 -- 172.21.15.55:0/3144552198 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec400bd90 con 0x7efecc06a670 2023-06-23T00:35:55.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.155+0000 7efec9d9b700 1 -- 172.21.15.55:0/3144552198 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec40116b0 con 0x7efecc06a670 2023-06-23T00:35:55.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.155+0000 7efed11aa700 1 -- 172.21.15.55:0/3144552198 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc06a670 msgr2=0x7efecc161340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.155+0000 7efed11aa700 1 --2- 172.21.15.55:0/3144552198 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc06a670 0x7efecc161340 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efec4009e90 tx=0x7efec4009780 comp rx=0 tx=0).stop 2023-06-23T00:35:55.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.155+0000 7efed11aa700 1 -- 172.21.15.55:0/3144552198 shutdown_connections 2023-06-23T00:35:55.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.155+0000 7efed11aa700 1 --2- 172.21.15.55:0/3144552198 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc06a670 0x7efecc161340 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.156+0000 7efed11aa700 1 -- 172.21.15.55:0/3144552198 >> 172.21.15.55:0/3144552198 conn(0x7efecc068520 msgr2=0x7efecc068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:55.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.156+0000 7efed11aa700 1 -- 172.21.15.55:0/3144552198 shutdown_connections 2023-06-23T00:35:55.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.156+0000 7efed11aa700 1 -- 172.21.15.55:0/3144552198 wait complete. 2023-06-23T00:35:55.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.157+0000 7efed11aa700 1 Processor -- start 2023-06-23T00:35:55.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.157+0000 7efed11aa700 1 -- start start 2023-06-23T00:35:55.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efed11aa700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efecc06a670 0x7efecc0969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efed11aa700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efecc159a40 0x7efecc0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efed11aa700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc159e10 0x7efecc148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efed11aa700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc157d30 con 0x7efecc06a670 2023-06-23T00:35:55.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efed11aa700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc097cd0 con 0x7efecc159a40 2023-06-23T00:35:55.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efed11aa700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0970c0 con 0x7efecc159e10 2023-06-23T00:35:55.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efecad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efecc06a670 0x7efecc0969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efecad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efecc06a670 0x7efecc0969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39506/0 (socket says 172.21.15.55:39506) 2023-06-23T00:35:55.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efecad9d700 1 -- 172.21.15.55:0/2566796994 learned_addr learned my addr 172.21.15.55:0/2566796994 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:55.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efecb59e700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc159e10 0x7efecc148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efeca59c700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efecc159a40 0x7efecc0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efecb59e700 1 -- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efecc159a40 msgr2=0x7efecc0c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efecb59e700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efecc159a40 0x7efecc0c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efecb59e700 1 -- 172.21.15.55:0/2566796994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efecc06a670 msgr2=0x7efecc0969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efecb59e700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efecc06a670 0x7efecc0969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efecb59e700 1 -- 172.21.15.55:0/2566796994 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4009a20 con 0x7efecc159e10 2023-06-23T00:35:55.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efeca59c700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efecc159a40 0x7efecc0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T00:35:55.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.158+0000 7efecb59e700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc159e10 0x7efecc148910 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efec4018000 tx=0x7efec401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:55.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.159+0000 7efec2ffd700 1 -- 172.21.15.55:0/2566796994 <== mon.2 v2:172.21.15.141:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4028070 con 0x7efecc159e10 2023-06-23T00:35:55.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.159+0000 7efec2ffd700 1 -- 172.21.15.55:0/2566796994 <== mon.2 v2:172.21.15.141:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec40032f0 con 0x7efecc159e10 2023-06-23T00:35:55.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.159+0000 7efed11aa700 1 -- 172.21.15.55:0/2566796994 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc0973a0 con 0x7efecc159e10 2023-06-23T00:35:55.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.159+0000 7efec2ffd700 1 -- 172.21.15.55:0/2566796994 <== mon.2 v2:172.21.15.141:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec400bc30 con 0x7efecc159e10 2023-06-23T00:35:55.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.159+0000 7efed11aa700 1 -- 172.21.15.55:0/2566796994 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc097960 con 0x7efecc159e10 2023-06-23T00:35:55.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.160+0000 7efed11aa700 1 -- 172.21.15.55:0/2566796994 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc04c1b0 con 0x7efecc159e10 2023-06-23T00:35:55.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.160+0000 7efec2ffd700 1 -- 172.21.15.55:0/2566796994 <== mon.2 v2:172.21.15.141:3301/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7efec4022030 con 0x7efecc159e10 2023-06-23T00:35:55.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.160+0000 7efec2ffd700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7efeac066e00 0x7efeac0692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.160+0000 7efec2ffd700 1 -- 172.21.15.55:0/2566796994 <== mon.2 v2:172.21.15.141:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7efec4087ba0 con 0x7efecc159e10 2023-06-23T00:35:55.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.161+0000 7efecad9d700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7efeac066e00 0x7efeac0692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.161+0000 7efecad9d700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7efeac066e00 0x7efeac0692b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efebc000c00 tx=0x7efebc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:55.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.165+0000 7efec2ffd700 1 -- 172.21.15.55:0/2566796994 <== mon.2 v2:172.21.15.141:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7efec4055100 con 0x7efecc159e10 2023-06-23T00:35:55.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.362+0000 7efed11aa700 1 -- 172.21.15.55:0/2566796994 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efecc097e10 con 0x7efecc159e10 2023-06-23T00:35:55.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.363+0000 7efec2ffd700 1 -- 172.21.15.55:0/2566796994 <== mon.2 v2:172.21.15.141:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7efec4058720 con 0x7efecc159e10 2023-06-23T00:35:55.367 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-06-23T00:35:55.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.365+0000 7efed11aa700 1 -- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7efeac066e00 msgr2=0x7efeac0692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.365+0000 7efed11aa700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7efeac066e00 0x7efeac0692b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efebc000c00 tx=0x7efebc009040 comp rx=0 tx=0).stop 2023-06-23T00:35:55.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.365+0000 7efed11aa700 1 -- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc159e10 msgr2=0x7efecc148910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.365+0000 7efed11aa700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc159e10 0x7efecc148910 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efec4018000 tx=0x7efec401b000 comp rx=0 tx=0).stop 2023-06-23T00:35:55.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.366+0000 7efed11aa700 1 -- 172.21.15.55:0/2566796994 shutdown_connections 2023-06-23T00:35:55.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.366+0000 7efed11aa700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7efeac066e00 0x7efeac0692b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.366+0000 7efed11aa700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7efecc159e10 0x7efecc148910 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.366+0000 7efed11aa700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efecc06a670 0x7efecc0969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.366+0000 7efed11aa700 1 --2- 172.21.15.55:0/2566796994 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7efecc159a40 0x7efecc0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.366+0000 7efed11aa700 1 -- 172.21.15.55:0/2566796994 >> 172.21.15.55:0/2566796994 conn(0x7efecc068520 msgr2=0x7efecc0c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:55.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.366+0000 7efed11aa700 1 -- 172.21.15.55:0/2566796994 shutdown_connections 2023-06-23T00:35:55.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.366+0000 7efed11aa700 1 -- 172.21.15.55:0/2566796994 wait complete. 2023-06-23T00:35:55.381 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T00:35:55.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.502+0000 7f28c5fd3700 1 Processor -- start 2023-06-23T00:35:55.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.502+0000 7f28c5fd3700 1 -- start start 2023-06-23T00:35:55.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.502+0000 7f28c5fd3700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28c0153a80 0x7f28c00d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.502+0000 7f28c5fd3700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f28c00b2010 0x7f28c00b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.502+0000 7f28c5fd3700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c00cedd0 con 0x7f28c0153e50 2023-06-23T00:35:55.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.502+0000 7f28c5fd3700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c00d14c0 con 0x7f28c0153a80 2023-06-23T00:35:55.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.502+0000 7f28c5fd3700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00b29d0 con 0x7f28c00b2010 2023-06-23T00:35:55.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28bf7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28c0153a80 0x7f28c00d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28bf7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28c0153a80 0x7f28c00d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39510/0 (socket says 172.21.15.55:39510) 2023-06-23T00:35:55.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28bf7fe700 1 -- 172.21.15.55:0/808357639 learned_addr learned my addr 172.21.15.55:0/808357639 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:55.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28bffff700 1 --2- 172.21.15.55:0/808357639 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f28c00b2010 0x7f28c00b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28bf7fe700 1 -- 172.21.15.55:0/808357639 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f28c00b2010 msgr2=0x7f28c00b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:55.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28bf7fe700 1 --2- 172.21.15.55:0/808357639 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f28c00b2010 0x7f28c00b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28bf7fe700 1 -- 172.21.15.55:0/808357639 >> v1:172.21.15.141:6789/0 conn(0x7f28c0153e50 legacy=0x7f28c015a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28bf7fe700 1 -- 172.21.15.55:0/808357639 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c015aed0 con 0x7f28c0153a80 2023-06-23T00:35:55.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28bf7fe700 1 --2- 172.21.15.55:0/808357639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153a80 0x7f28c00d0590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009f50 tx=0x7f28b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e72b2565e6c2e2a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:55.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28be7fc700 1 -- 172.21.15.55:0/808357639 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b800e070 con 0x7f28c0153a80 2023-06-23T00:35:55.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28be7fc700 1 -- 172.21.15.55:0/808357639 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b800b9c0 con 0x7f28c0153a80 2023-06-23T00:35:55.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.503+0000 7f28be7fc700 1 -- 172.21.15.55:0/808357639 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80092e0 con 0x7f28c0153a80 2023-06-23T00:35:55.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.504+0000 7f28c5fd3700 1 -- 172.21.15.55:0/808357639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153a80 msgr2=0x7f28c00d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.504+0000 7f28c5fd3700 1 --2- 172.21.15.55:0/808357639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153a80 0x7f28c00d0590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009f50 tx=0x7f28b8009840 comp rx=0 tx=0).stop 2023-06-23T00:35:55.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.504+0000 7f28c5fd3700 1 -- 172.21.15.55:0/808357639 shutdown_connections 2023-06-23T00:35:55.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.504+0000 7f28c5fd3700 1 --2- 172.21.15.55:0/808357639 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f28c00b2010 0x7f28c00b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.504+0000 7f28c5fd3700 1 --2- 172.21.15.55:0/808357639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153a80 0x7f28c00d0590 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.504+0000 7f28c5fd3700 1 -- 172.21.15.55:0/808357639 >> 172.21.15.55:0/808357639 conn(0x7f28c0070290 msgr2=0x7f28c0070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:55.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.504+0000 7f28c5fd3700 1 -- 172.21.15.55:0/808357639 shutdown_connections 2023-06-23T00:35:55.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.504+0000 7f28c5fd3700 1 -- 172.21.15.55:0/808357639 wait complete. 2023-06-23T00:35:55.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.505+0000 7f28c5fd3700 1 Processor -- start 2023-06-23T00:35:55.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.505+0000 7f28c5fd3700 1 -- start start 2023-06-23T00:35:55.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28c5fd3700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f28c00b2010 0x7f28c00cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28c5fd3700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f28c0153a80 0x7f28c0155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28c5fd3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153e50 0x7f28c016a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28c5fd3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c0169ef0 con 0x7f28c0153e50 2023-06-23T00:35:55.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28c5fd3700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00cc0a0 con 0x7f28c00b2010 2023-06-23T00:35:55.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28c5fd3700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c016d910 con 0x7f28c0153a80 2023-06-23T00:35:55.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28bffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153e50 0x7f28c016a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28bffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153e50 0x7f28c016a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39526/0 (socket says 172.21.15.55:39526) 2023-06-23T00:35:55.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28beffd700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f28c0153a80 0x7f28c0155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28bf7fe700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f28c00b2010 0x7f28c00cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28bffff700 1 -- 172.21.15.55:0/3242283782 learned_addr learned my addr 172.21.15.55:0/3242283782 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:55.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28bffff700 1 -- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f28c0153a80 msgr2=0x7f28c0155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28bffff700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f28c0153a80 0x7f28c0155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28bffff700 1 -- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f28c00b2010 msgr2=0x7f28c00cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28bffff700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f28c00b2010 0x7f28c00cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28bffff700 1 -- 172.21.15.55:0/3242283782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8009ae0 con 0x7f28c0153e50 2023-06-23T00:35:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28bffff700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153e50 0x7f28c016a380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f28b000e8d0 tx=0x7f28b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.506+0000 7f28b77fe700 1 -- 172.21.15.55:0/3242283782 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0013680 con 0x7f28c0153e50 2023-06-23T00:35:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.507+0000 7f28b77fe700 1 -- 172.21.15.55:0/3242283782 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b00137e0 con 0x7f28c0153e50 2023-06-23T00:35:55.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.507+0000 7f28c5fd3700 1 -- 172.21.15.55:0/3242283782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c016dbf0 con 0x7f28c0153e50 2023-06-23T00:35:55.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.507+0000 7f28b77fe700 1 -- 172.21.15.55:0/3242283782 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0013ba0 con 0x7f28c0153e50 2023-06-23T00:35:55.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.507+0000 7f28c5fd3700 1 -- 172.21.15.55:0/3242283782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c016e1b0 con 0x7f28c0153e50 2023-06-23T00:35:55.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.508+0000 7f28c5fd3700 1 -- 172.21.15.55:0/3242283782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c016a0a0 con 0x7f28c0153e50 2023-06-23T00:35:55.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.508+0000 7f28b77fe700 1 -- 172.21.15.55:0/3242283782 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f28b00123d0 con 0x7f28c0153e50 2023-06-23T00:35:55.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.508+0000 7f28b77fe700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f28a0066ec0 0x7f28a0069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.509+0000 7f28b77fe700 1 -- 172.21.15.55:0/3242283782 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f28b0084300 con 0x7f28c0153e50 2023-06-23T00:35:55.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.509+0000 7f28bf7fe700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f28a0066ec0 0x7f28a0069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.509+0000 7f28bf7fe700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f28a0066ec0 0x7f28a0069370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009ea0 tx=0x7f28b8018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.513+0000 7f28b77fe700 1 -- 172.21.15.55:0/3242283782 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f28b0051820 con 0x7f28c0153e50 2023-06-23T00:35:55.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.708+0000 7f28c5fd3700 1 -- 172.21.15.55:0/3242283782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28c004c1b0 con 0x7f28c0153e50 2023-06-23T00:35:55.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.708+0000 7f28b77fe700 1 -- 172.21.15.55:0/3242283782 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28b0054e40 con 0x7f28c0153e50 2023-06-23T00:35:55.713 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-06-23T00:35:55.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.710+0000 7f28c5fd3700 1 -- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f28a0066ec0 msgr2=0x7f28a0069370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.710+0000 7f28c5fd3700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f28a0066ec0 0x7f28a0069370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009ea0 tx=0x7f28b8018000 comp rx=0 tx=0).stop 2023-06-23T00:35:55.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.710+0000 7f28c5fd3700 1 -- 172.21.15.55:0/3242283782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153e50 msgr2=0x7f28c016a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.710+0000 7f28c5fd3700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153e50 0x7f28c016a380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f28b000e8d0 tx=0x7f28b00091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.710+0000 7f28c5fd3700 1 -- 172.21.15.55:0/3242283782 shutdown_connections 2023-06-23T00:35:55.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.710+0000 7f28c5fd3700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f28a0066ec0 0x7f28a0069370 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.710+0000 7f28c5fd3700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28c0153e50 0x7f28c016a380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.710+0000 7f28c5fd3700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f28c00b2010 0x7f28c00cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.710+0000 7f28c5fd3700 1 --2- 172.21.15.55:0/3242283782 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f28c0153a80 0x7f28c0155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.710+0000 7f28c5fd3700 1 -- 172.21.15.55:0/3242283782 >> 172.21.15.55:0/3242283782 conn(0x7f28c0070290 msgr2=0x7f28c015b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:55.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.711+0000 7f28c5fd3700 1 -- 172.21.15.55:0/3242283782 shutdown_connections 2023-06-23T00:35:55.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.711+0000 7f28c5fd3700 1 -- 172.21.15.55:0/3242283782 wait complete. 2023-06-23T00:35:55.727 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-23T00:35:55.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.842+0000 7f7556f55700 1 Processor -- start 2023-06-23T00:35:55.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.842+0000 7f7556f55700 1 -- start start 2023-06-23T00:35:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.843+0000 7f7556f55700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f75500ae700 0x7f75500c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.843+0000 7f7556f55700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75500ade90 con 0x7f7550156320 2023-06-23T00:35:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.843+0000 7f7556f55700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75500c69a0 con 0x7f75500ae280 2023-06-23T00:35:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.843+0000 7f7556f55700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75500ad5f0 con 0x7f75500ae700 2023-06-23T00:35:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.843+0000 7f7554cf1700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7550156320 0x7f75500acaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56934/0 (socket says 172.21.15.55:56934) 2023-06-23T00:35:55.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.843+0000 7f7554cf1700 1 -- 172.21.15.55:0/2323031140 learned_addr learned my addr 172.21.15.55:0/2323031140 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:55.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.843+0000 7f75554f2700 1 --2- 172.21.15.55:0/2323031140 >> v2:172.21.15.141:3300/0 conn(0x7f75500ae700 0x7f75500c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.843+0000 7f754f7fe700 1 -- 172.21.15.55:0/2323031140 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111815606 0 0) 0x7f75500ade90 con 0x7f7550156320 2023-06-23T00:35:55.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f754f7fe700 1 -- 172.21.15.55:0/2323031140 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7538003570 con 0x7f7550156320 2023-06-23T00:35:55.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f754f7fe700 1 -- 172.21.15.55:0/2323031140 <== mon.1 v1:172.21.15.141:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176143618 0 0) 0x7f75500c69a0 con 0x7f75500ae280 2023-06-23T00:35:55.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f754f7fe700 1 -- 172.21.15.55:0/2323031140 --> v1:172.21.15.141:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75500ade90 con 0x7f75500ae280 2023-06-23T00:35:55.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f75554f2700 1 -- 172.21.15.55:0/2323031140 >> v1:172.21.15.141:6789/0 conn(0x7f75500ae280 legacy=0x7f75500c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f75554f2700 1 -- 172.21.15.55:0/2323031140 >> v1:172.21.15.55:6789/0 conn(0x7f7550156320 legacy=0x7f75500acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f75554f2700 1 -- 172.21.15.55:0/2323031140 --> v2:172.21.15.141:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500ad8d0 con 0x7f75500ae700 2023-06-23T00:35:55.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f75554f2700 1 --2- 172.21.15.55:0/2323031140 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f75500ae700 0x7f75500c7540 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7548009e60 tx=0x7f7548009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c0897cfa04044e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:55.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f754f7fe700 1 -- 172.21.15.55:0/2323031140 <== mon.1 v2:172.21.15.141:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800e070 con 0x7f75500ae700 2023-06-23T00:35:55.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f754f7fe700 1 -- 172.21.15.55:0/2323031140 <== mon.1 v2:172.21.15.141:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754800bab0 con 0x7f75500ae700 2023-06-23T00:35:55.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f754f7fe700 1 -- 172.21.15.55:0/2323031140 <== mon.1 v2:172.21.15.141:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800be70 con 0x7f75500ae700 2023-06-23T00:35:55.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f7556f55700 1 -- 172.21.15.55:0/2323031140 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f75500ae700 msgr2=0x7f75500c7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.844+0000 7f7556f55700 1 --2- 172.21.15.55:0/2323031140 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f75500ae700 0x7f75500c7540 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7548009e60 tx=0x7f7548009780 comp rx=0 tx=0).stop 2023-06-23T00:35:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.845+0000 7f7556f55700 1 -- 172.21.15.55:0/2323031140 shutdown_connections 2023-06-23T00:35:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.845+0000 7f7556f55700 1 --2- 172.21.15.55:0/2323031140 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f75500ae700 0x7f75500c7540 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.845+0000 7f7556f55700 1 -- 172.21.15.55:0/2323031140 >> 172.21.15.55:0/2323031140 conn(0x7f7550067410 msgr2=0x7f7550067810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.845+0000 7f7556f55700 1 -- 172.21.15.55:0/2323031140 shutdown_connections 2023-06-23T00:35:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.845+0000 7f7556f55700 1 -- 172.21.15.55:0/2323031140 wait complete. 2023-06-23T00:35:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.846+0000 7f7556f55700 1 Processor -- start 2023-06-23T00:35:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.846+0000 7f7556f55700 1 -- start start 2023-06-23T00:35:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.846+0000 7f7556f55700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75500ae280 0x7f7550156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.846+0000 7f7556f55700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f75500ae700 0x7f7550159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7556f55700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f7550156320 0x7f75500b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7556f55700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7550156cc0 con 0x7f75500ae280 2023-06-23T00:35:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7556f55700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7550157ae0 con 0x7f75500ae700 2023-06-23T00:35:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7556f55700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7550157800 con 0x7f7550156320 2023-06-23T00:35:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7554cf1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75500ae280 0x7f7550156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7554cf1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75500ae280 0x7f7550156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39532/0 (socket says 172.21.15.55:39532) 2023-06-23T00:35:55.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7554cf1700 1 -- 172.21.15.55:0/972256300 learned_addr learned my addr 172.21.15.55:0/972256300 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:55.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f75554f2700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f7550156320 0x7f75500b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f754ffff700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f75500ae700 0x7f7550159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7554cf1700 1 -- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f7550156320 msgr2=0x7f75500b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7554cf1700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f7550156320 0x7f75500b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7554cf1700 1 -- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f75500ae700 msgr2=0x7f7550159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:55.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7554cf1700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f75500ae700 0x7f7550159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7554cf1700 1 -- 172.21.15.55:0/972256300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7548009a20 con 0x7f75500ae280 2023-06-23T00:35:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7554cf1700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75500ae280 0x7f7550156780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f754000c7c0 tx=0x7f7540007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f754cff9700 1 -- 172.21.15.55:0/972256300 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540005b30 con 0x7f75500ae280 2023-06-23T00:35:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f754cff9700 1 -- 172.21.15.55:0/972256300 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7540005c90 con 0x7f75500ae280 2023-06-23T00:35:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7556f55700 1 -- 172.21.15.55:0/972256300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7550159640 con 0x7f75500ae280 2023-06-23T00:35:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f754cff9700 1 -- 172.21.15.55:0/972256300 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540007300 con 0x7f75500ae280 2023-06-23T00:35:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.847+0000 7f7556f55700 1 -- 172.21.15.55:0/972256300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7550157230 con 0x7f75500ae280 2023-06-23T00:35:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.848+0000 7f7556f55700 1 -- 172.21.15.55:0/972256300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75501674d0 con 0x7f75500ae280 2023-06-23T00:35:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.849+0000 7f754cff9700 1 -- 172.21.15.55:0/972256300 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f754001b3b0 con 0x7f75500ae280 2023-06-23T00:35:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.849+0000 7f754cff9700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f753c066ec0 0x7f753c069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:55.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.849+0000 7f754cff9700 1 -- 172.21.15.55:0/972256300 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7540083660 con 0x7f75500ae280 2023-06-23T00:35:55.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.849+0000 7f754ffff700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f753c066ec0 0x7f753c069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:55.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.850+0000 7f754ffff700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f753c066ec0 0x7f753c069370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f75500c3090 tx=0x7f7544007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:55.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:55.854+0000 7f754cff9700 1 -- 172.21.15.55:0/972256300 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f7540050b80 con 0x7f75500ae280 2023-06-23T00:35:56.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.051+0000 7f7556f55700 1 -- 172.21.15.55:0/972256300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f755004c1b0 con 0x7f75500ae280 2023-06-23T00:35:56.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.051+0000 7f754cff9700 1 -- 172.21.15.55:0/972256300 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7540017090 con 0x7f75500ae280 2023-06-23T00:35:56.056 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:56.056 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-06-23T00:35:56.056 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-06-23T00:35:56.057 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-06-23T00:35:56.057 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-06-23T00:35:56.057 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-06-23T00:35:56.057 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-06-23T00:35:56.057 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-06-23T00:35:56.058 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-06-23T00:35:56.058 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-06-23T00:35:56.058 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-06-23T00:35:56.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 -- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f753c066ec0 msgr2=0x7f753c069370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:56.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f753c066ec0 0x7f753c069370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f75500c3090 tx=0x7f7544007d50 comp rx=0 tx=0).stop 2023-06-23T00:35:56.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 -- 172.21.15.55:0/972256300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75500ae280 msgr2=0x7f7550156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:56.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75500ae280 0x7f7550156780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f754000c7c0 tx=0x7f7540007ea0 comp rx=0 tx=0).stop 2023-06-23T00:35:56.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 -- 172.21.15.55:0/972256300 shutdown_connections 2023-06-23T00:35:56.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f753c066ec0 0x7f753c069370 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:56.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f7550156320 0x7f75500b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:56.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75500ae280 0x7f7550156780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:56.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 --2- 172.21.15.55:0/972256300 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f75500ae700 0x7f7550159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:56.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 -- 172.21.15.55:0/972256300 >> 172.21.15.55:0/972256300 conn(0x7f7550067410 msgr2=0x7f75500c7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:56.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 -- 172.21.15.55:0/972256300 shutdown_connections 2023-06-23T00:35:56.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.053+0000 7f7556f55700 1 -- 172.21.15.55:0/972256300 wait complete. 2023-06-23T00:35:56.070 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-23T00:35:56.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-23T00:35:56.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.190+0000 7f714cfa7700 1 Processor -- start 2023-06-23T00:35:56.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.190+0000 7f714cfa7700 1 -- start start 2023-06-23T00:35:56.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.191+0000 7f714cfa7700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7148159320 0x7f71481596f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:56.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.191+0000 7f714cfa7700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71480ce9d0 con 0x7f71480d0bb0 2023-06-23T00:35:56.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.191+0000 7f714cfa7700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71480cec00 con 0x7f71480ccfd0 2023-06-23T00:35:56.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.191+0000 7f714cfa7700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71480ceee0 con 0x7f7148159320 2023-06-23T00:35:56.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.191+0000 7f714659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7148159320 0x7f71481596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:56.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.192+0000 7f7146d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f71480d0bb0 0x7f71480b3e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56940/0 (socket says 172.21.15.55:56940) 2023-06-23T00:35:56.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.192+0000 7f714659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7148159320 0x7f71481596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39546/0 (socket says 172.21.15.55:39546) 2023-06-23T00:35:56.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.192+0000 7f7146d9d700 1 -- 172.21.15.55:0/3904411375 learned_addr learned my addr 172.21.15.55:0/3904411375 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:56.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.192+0000 7f714659c700 1 -- 172.21.15.55:0/3904411375 >> v1:172.21.15.141:6789/0 conn(0x7f71480ccfd0 legacy=0x7f71480d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:56.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.192+0000 7f714659c700 1 -- 172.21.15.55:0/3904411375 >> v1:172.21.15.55:6789/0 conn(0x7f71480d0bb0 legacy=0x7f71480b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:56.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.192+0000 7f714659c700 1 -- 172.21.15.55:0/3904411375 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71480cf1c0 con 0x7f7148159320 2023-06-23T00:35:56.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.192+0000 7f714659c700 1 --2- 172.21.15.55:0/3904411375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7148159320 0x7f71481596f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7140009e60 tx=0x7f7140009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956f442da20f627a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:56.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.192+0000 7f714559a700 1 -- 172.21.15.55:0/3904411375 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714000e070 con 0x7f7148159320 2023-06-23T00:35:56.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.192+0000 7f714559a700 1 -- 172.21.15.55:0/3904411375 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714000bc30 con 0x7f7148159320 2023-06-23T00:35:56.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.192+0000 7f714559a700 1 -- 172.21.15.55:0/3904411375 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140012430 con 0x7f7148159320 2023-06-23T00:35:56.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.193+0000 7f714cfa7700 1 -- 172.21.15.55:0/3904411375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7148159320 msgr2=0x7f71481596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:56.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.193+0000 7f714cfa7700 1 --2- 172.21.15.55:0/3904411375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7148159320 0x7f71481596f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7140009e60 tx=0x7f7140009780 comp rx=0 tx=0).stop 2023-06-23T00:35:56.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.193+0000 7f714cfa7700 1 -- 172.21.15.55:0/3904411375 shutdown_connections 2023-06-23T00:35:56.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.193+0000 7f714cfa7700 1 --2- 172.21.15.55:0/3904411375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7148159320 0x7f71481596f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:56.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.193+0000 7f714cfa7700 1 -- 172.21.15.55:0/3904411375 >> 172.21.15.55:0/3904411375 conn(0x7f7148070290 msgr2=0x7f7148070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:56.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.193+0000 7f714cfa7700 1 -- 172.21.15.55:0/3904411375 shutdown_connections 2023-06-23T00:35:56.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.193+0000 7f714cfa7700 1 -- 172.21.15.55:0/3904411375 wait complete. 2023-06-23T00:35:56.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.194+0000 7f714cfa7700 1 Processor -- start 2023-06-23T00:35:56.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.194+0000 7f714cfa7700 1 -- start start 2023-06-23T00:35:56.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f714cfa7700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f71480ccfd0 0x7f7148150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:56.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f714cfa7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71480d0bb0 0x7f714814e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:56.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f714cfa7700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7148159320 0x7f7148169e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:56.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f714cfa7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714814f5c0 con 0x7f71480d0bb0 2023-06-23T00:35:56.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f714cfa7700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714816b200 con 0x7f7148159320 2023-06-23T00:35:56.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f714cfa7700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714814ca30 con 0x7f71480ccfd0 2023-06-23T00:35:56.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f7145d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71480d0bb0 0x7f714814e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:56.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f7145d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71480d0bb0 0x7f714814e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39562/0 (socket says 172.21.15.55:39562) 2023-06-23T00:35:56.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f7145d9b700 1 -- 172.21.15.55:0/1406060341 learned_addr learned my addr 172.21.15.55:0/1406060341 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:56.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f7146d9d700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7148159320 0x7f7148169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:56.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f714659c700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f71480ccfd0 0x7f7148150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:56.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f7145d9b700 1 -- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f71480ccfd0 msgr2=0x7f7148150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:56.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f7145d9b700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f71480ccfd0 0x7f7148150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:56.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f7145d9b700 1 -- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7148159320 msgr2=0x7f7148169e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:56.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f7145d9b700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7148159320 0x7f7148169e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:56.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f7145d9b700 1 -- 172.21.15.55:0/1406060341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7140009a20 con 0x7f71480d0bb0 2023-06-23T00:35:56.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f7145d9b700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71480d0bb0 0x7f714814e540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f713000c7c0 tx=0x7f7130007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:56.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f713e7fc700 1 -- 172.21.15.55:0/1406060341 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130011590 con 0x7f71480d0bb0 2023-06-23T00:35:56.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f713e7fc700 1 -- 172.21.15.55:0/1406060341 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7130011710 con 0x7f71480d0bb0 2023-06-23T00:35:56.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f714cfa7700 1 -- 172.21.15.55:0/1406060341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714814ea80 con 0x7f71480d0bb0 2023-06-23T00:35:56.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f713e7fc700 1 -- 172.21.15.55:0/1406060341 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130011ad0 con 0x7f71480d0bb0 2023-06-23T00:35:56.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.195+0000 7f714cfa7700 1 -- 172.21.15.55:0/1406060341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714814f000 con 0x7f71480d0bb0 2023-06-23T00:35:56.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.196+0000 7f714cfa7700 1 -- 172.21.15.55:0/1406060341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714804c1b0 con 0x7f71480d0bb0 2023-06-23T00:35:56.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.197+0000 7f713e7fc700 1 -- 172.21.15.55:0/1406060341 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f713001b3b0 con 0x7f71480d0bb0 2023-06-23T00:35:56.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.197+0000 7f713e7fc700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7128066e90 0x7f7128069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:56.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.197+0000 7f713e7fc700 1 -- 172.21.15.55:0/1406060341 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f71300835e0 con 0x7f71480d0bb0 2023-06-23T00:35:56.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.197+0000 7f714659c700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7128066e90 0x7f7128069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:56.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.197+0000 7f714659c700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7128066e90 0x7f7128069340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7140009e60 tx=0x7f7140019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:56.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.201+0000 7f713e7fc700 1 -- 172.21.15.55:0/1406060341 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f7130050b00 con 0x7f71480d0bb0 2023-06-23T00:35:56.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:56.393+0000 7f714cfa7700 1 -- 172.21.15.55:0/1406060341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f714816b340 con 0x7f71480d0bb0 2023-06-23T00:35:57.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.315+0000 7f713e7fc700 1 -- 172.21.15.55:0/1406060341 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7130017070 con 0x7f71480d0bb0 2023-06-23T00:35:57.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 -- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7128066e90 msgr2=0x7f7128069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:57.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7128066e90 0x7f7128069340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7140009e60 tx=0x7f7140019000 comp rx=0 tx=0).stop 2023-06-23T00:35:57.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 -- 172.21.15.55:0/1406060341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71480d0bb0 msgr2=0x7f714814e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:57.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71480d0bb0 0x7f714814e540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f713000c7c0 tx=0x7f7130007f40 comp rx=0 tx=0).stop 2023-06-23T00:35:57.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 -- 172.21.15.55:0/1406060341 shutdown_connections 2023-06-23T00:35:57.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7128066e90 0x7f7128069340 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7140009e60 tx=0x7f7140019000 comp rx=0 tx=0).stop 2023-06-23T00:35:57.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7148159320 0x7f7148169e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:57.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f71480ccfd0 0x7f7148150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:57.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 --2- 172.21.15.55:0/1406060341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71480d0bb0 0x7f714814e540 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:57.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 -- 172.21.15.55:0/1406060341 >> 172.21.15.55:0/1406060341 conn(0x7f7148070290 msgr2=0x7f714814d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:57.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 -- 172.21.15.55:0/1406060341 shutdown_connections 2023-06-23T00:35:57.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.319+0000 7f714cfa7700 1 -- 172.21.15.55:0/1406060341 wait complete. 2023-06-23T00:35:57.329 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-06-23T00:35:57.341 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-23T00:35:57.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.469+0000 7fe1d1034700 1 Processor -- start 2023-06-23T00:35:57.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.470+0000 7fe1d1034700 1 -- start start 2023-06-23T00:35:57.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.470+0000 7fe1d1034700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7fe1cc153a80 0x7fe1cc0d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:57.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.470+0000 7fe1d1034700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1cc0cedd0 con 0x7fe1cc153e50 2023-06-23T00:35:57.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.470+0000 7fe1d1034700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1cc15aed0 con 0x7fe1cc0b2010 2023-06-23T00:35:57.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.470+0000 7fe1d1034700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc0d14c0 con 0x7fe1cc153a80 2023-06-23T00:35:57.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.470+0000 7fe1ca59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe1cc153e50 0x7fe1cc15a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56946/0 (socket says 172.21.15.55:56946) 2023-06-23T00:35:57.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1ca59c700 1 -- 172.21.15.55:0/4012211518 learned_addr learned my addr 172.21.15.55:0/4012211518 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:57.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1cad9d700 1 --2- 172.21.15.55:0/4012211518 >> v2:172.21.15.141:3300/0 conn(0x7fe1cc153a80 0x7fe1cc0d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:57.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573734322 0 0) 0x7fe1cc0cedd0 con 0x7fe1cc153e50 2023-06-23T00:35:57.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1b8003570 con 0x7fe1cc153e50 2023-06-23T00:35:57.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 <== mon.1 v1:172.21.15.141:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253248539 0 0) 0x7fe1cc15aed0 con 0x7fe1cc0b2010 2023-06-23T00:35:57.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 --> v1:172.21.15.141:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1cc0cedd0 con 0x7fe1cc0b2010 2023-06-23T00:35:57.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2947164413 0 0) 0x7fe1b8003570 con 0x7fe1cc153e50 2023-06-23T00:35:57.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 >> v2:172.21.15.141:3300/0 conn(0x7fe1cc153a80 msgr2=0x7fe1cc0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:57.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 --2- 172.21.15.55:0/4012211518 >> v2:172.21.15.141:3300/0 conn(0x7fe1cc153a80 0x7fe1cc0d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:57.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 >> v1:172.21.15.141:6789/0 conn(0x7fe1cc0b2010 legacy=0x7fe1cc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:57.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc0ceac0 con 0x7fe1cc153e50 2023-06-23T00:35:57.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2542258539 0 0) 0x7fe1b4001470 con 0x7fe1cc153e50 2023-06-23T00:35:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 >> v1:172.21.15.55:6789/0 conn(0x7fe1cc153e50 legacy=0x7fe1cc15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe1b8003c40 0x7fe1b8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1b8004810 0x7fe1b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8005210 0x7fe1b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0d14c0 con 0x7fe1b8005210 2023-06-23T00:35:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b800c1b0 con 0x7fe1b8004810 2023-06-23T00:35:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b800c370 con 0x7fe1b8003c40 2023-06-23T00:35:57.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1cb59e700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8005210 0x7fe1b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:57.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1cad9d700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe1b8003c40 0x7fe1b8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:57.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.471+0000 7fe1ca59c700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1b8004810 0x7fe1b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:57.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1cb59e700 1 -- 172.21.15.55:0/4012211518 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe1b8003c40 msgr2=0x7fe1b8004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:57.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1cb59e700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe1b8003c40 0x7fe1b8004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:57.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1cb59e700 1 -- 172.21.15.55:0/4012211518 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1b8004810 msgr2=0x7fe1b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:57.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1cb59e700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1b8004810 0x7fe1b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:57.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1cb59e700 1 -- 172.21.15.55:0/4012211518 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b8003570 con 0x7fe1b8005210 2023-06-23T00:35:57.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1ca59c700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1b8004810 0x7fe1b8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T00:35:57.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1cb59e700 1 -- 172.21.15.55:0/4012211518 reap_dead start 2023-06-23T00:35:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1cb59e700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8005210 0x7fe1b800ba60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0019e0 tx=0x7fe1bc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cda886a0afc08aa server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00c5d0 con 0x7fe1b8005210 2023-06-23T00:35:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc00c730 con 0x7fe1b8005210 2023-06-23T00:35:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1c9d9b700 1 -- 172.21.15.55:0/4012211518 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00cad0 con 0x7fe1b8005210 2023-06-23T00:35:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1d1034700 1 -- 172.21.15.55:0/4012211518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8005210 msgr2=0x7fe1b800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1d1034700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8005210 0x7fe1b800ba60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0019e0 tx=0x7fe1bc007d30 comp rx=0 tx=0).stop 2023-06-23T00:35:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1d1034700 1 -- 172.21.15.55:0/4012211518 shutdown_connections 2023-06-23T00:35:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1d1034700 1 --2- 172.21.15.55:0/4012211518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8005210 0x7fe1b800ba60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1d1034700 1 -- 172.21.15.55:0/4012211518 >> 172.21.15.55:0/4012211518 conn(0x7fe1cc070290 msgr2=0x7fe1cc070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1d1034700 1 -- 172.21.15.55:0/4012211518 shutdown_connections 2023-06-23T00:35:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.472+0000 7fe1d1034700 1 -- 172.21.15.55:0/4012211518 wait complete. 2023-06-23T00:35:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1d1034700 1 Processor -- start 2023-06-23T00:35:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1d1034700 1 -- start start 2023-06-23T00:35:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1d1034700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe1b8005210 0x7fe1cc157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1d1034700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1cc1568b0 0x7fe1cc169850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1d1034700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1cc169d90 0x7fe1cc0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1d1034700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc158ff0 con 0x7fe1cc169d90 2023-06-23T00:35:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1d1034700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc157e20 con 0x7fe1cc1568b0 2023-06-23T00:35:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1d1034700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc158100 con 0x7fe1b8005210 2023-06-23T00:35:57.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1cb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1cc169d90 0x7fe1cc0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:57.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1cb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1cc169d90 0x7fe1cc0b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39592/0 (socket says 172.21.15.55:39592) 2023-06-23T00:35:57.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.474+0000 7fe1cb59e700 1 -- 172.21.15.55:0/1631752893 learned_addr learned my addr 172.21.15.55:0/1631752893 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:57.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1ca59c700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1cc1568b0 0x7fe1cc169850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:57.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1cad9d700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe1b8005210 0x7fe1cc157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:57.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1cb59e700 1 -- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe1b8005210 msgr2=0x7fe1cc157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:57.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1cb59e700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe1b8005210 0x7fe1cc157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:57.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1cb59e700 1 -- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1cc1568b0 msgr2=0x7fe1cc169850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:57.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1cb59e700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1cc1568b0 0x7fe1cc169850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:57.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1cb59e700 1 -- 172.21.15.55:0/1631752893 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc1583e0 con 0x7fe1cc169d90 2023-06-23T00:35:57.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1ca59c700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1cc1568b0 0x7fe1cc169850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T00:35:57.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1cb59e700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1cc169d90 0x7fe1cc0b4ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc06f6a0 tx=0x7fe1bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:57.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1c2ffd700 1 -- 172.21.15.55:0/1631752893 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc001360 con 0x7fe1cc169d90 2023-06-23T00:35:57.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1c2ffd700 1 -- 172.21.15.55:0/1631752893 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc00ce80 con 0x7fe1cc169d90 2023-06-23T00:35:57.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1d1034700 1 -- 172.21.15.55:0/1631752893 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc1586c0 con 0x7fe1cc169d90 2023-06-23T00:35:57.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1c2ffd700 1 -- 172.21.15.55:0/1631752893 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc006a70 con 0x7fe1cc169d90 2023-06-23T00:35:57.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.475+0000 7fe1d1034700 1 -- 172.21.15.55:0/1631752893 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc0b5000 con 0x7fe1cc169d90 2023-06-23T00:35:57.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.476+0000 7fe1d1034700 1 -- 172.21.15.55:0/1631752893 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc16a200 con 0x7fe1cc169d90 2023-06-23T00:35:57.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.476+0000 7fe1c2ffd700 1 -- 172.21.15.55:0/1631752893 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fe1bc007030 con 0x7fe1cc169d90 2023-06-23T00:35:57.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.477+0000 7fe1c2ffd700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fe1ac066e90 0x7fe1ac069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:57.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.477+0000 7fe1c2ffd700 1 -- 172.21.15.55:0/1631752893 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fe1bc085280 con 0x7fe1cc169d90 2023-06-23T00:35:57.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.477+0000 7fe1cad9d700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fe1ac066e90 0x7fe1ac069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:57.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.477+0000 7fe1cad9d700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fe1ac066e90 0x7fe1ac069340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40094d0 tx=0x7fe1c400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:57.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.481+0000 7fe1c2ffd700 1 -- 172.21.15.55:0/1631752893 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fe1cc16a200 con 0x7fe1cc169d90 2023-06-23T00:35:57.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:57.676+0000 7fe1d1034700 1 -- 172.21.15.55:0/1631752893 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe1cc16a200 con 0x7fe1cc169d90 2023-06-23T00:35:58.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.323+0000 7fe1c2ffd700 1 -- 172.21.15.55:0/1631752893 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe1cc16a200 con 0x7fe1cc169d90 2023-06-23T00:35:58.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.328+0000 7fe1d1034700 1 -- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fe1ac066e90 msgr2=0x7fe1ac069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.328+0000 7fe1d1034700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fe1ac066e90 0x7fe1ac069340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40094d0 tx=0x7fe1c400b040 comp rx=0 tx=0).stop 2023-06-23T00:35:58.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.328+0000 7fe1d1034700 1 -- 172.21.15.55:0/1631752893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1cc169d90 msgr2=0x7fe1cc0b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.328+0000 7fe1d1034700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1cc169d90 0x7fe1cc0b4ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc06f6a0 tx=0x7fe1bc014000 comp rx=0 tx=0).stop 2023-06-23T00:35:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.329+0000 7fe1d1034700 1 -- 172.21.15.55:0/1631752893 shutdown_connections 2023-06-23T00:35:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.329+0000 7fe1d1034700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fe1ac066e90 0x7fe1ac069340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.329+0000 7fe1d1034700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1cc169d90 0x7fe1cc0b4ac0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.329+0000 7fe1d1034700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fe1b8005210 0x7fe1cc157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.329+0000 7fe1d1034700 1 --2- 172.21.15.55:0/1631752893 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fe1cc1568b0 0x7fe1cc169850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.329+0000 7fe1d1034700 1 -- 172.21.15.55:0/1631752893 >> 172.21.15.55:0/1631752893 conn(0x7fe1cc070290 msgr2=0x7fe1cc152290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:58.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.330+0000 7fe1d1034700 1 -- 172.21.15.55:0/1631752893 shutdown_connections 2023-06-23T00:35:58.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.330+0000 7fe1d1034700 1 -- 172.21.15.55:0/1631752893 wait complete. 2023-06-23T00:35:58.346 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-06-23T00:35:58.356 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-23T00:35:58.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.478+0000 7f1ce2e6a700 1 Processor -- start 2023-06-23T00:35:58.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.478+0000 7f1ce2e6a700 1 -- start start 2023-06-23T00:35:58.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.479+0000 7f1ce2e6a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1cdc0b41d0 0x7f1cdc0b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:58.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.479+0000 7f1ce2e6a700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc0ce0b0 con 0x7f1cdc0aebc0 2023-06-23T00:35:58.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.479+0000 7f1ce2e6a700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc0cd4a0 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.479+0000 7f1ce2e6a700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc0cd780 con 0x7f1cdc0b41d0 2023-06-23T00:35:58.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.479+0000 7f1cdbfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1cdc0b41d0 0x7f1cdc0b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:58.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.479+0000 7f1ce1407700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1cdc0aebc0 0x7f1cdc0b1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56954/0 (socket says 172.21.15.55:56954) 2023-06-23T00:35:58.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.479+0000 7f1ce1407700 1 -- 172.21.15.55:0/2922612390 learned_addr learned my addr 172.21.15.55:0/2922612390 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:58.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.480+0000 7f1cdbfff700 1 -- 172.21.15.55:0/2922612390 >> v1:172.21.15.141:6789/0 conn(0x7f1cdc0b3e00 legacy=0x7f1cdc0ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.480+0000 7f1cdbfff700 1 -- 172.21.15.55:0/2922612390 >> v1:172.21.15.55:6789/0 conn(0x7f1cdc0aebc0 legacy=0x7f1cdc0b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.480+0000 7f1cdbfff700 1 -- 172.21.15.55:0/2922612390 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc0cda60 con 0x7f1cdc0b41d0 2023-06-23T00:35:58.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.480+0000 7f1cdb7fe700 1 -- 172.21.15.55:0/2922612390 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182586998 0 0) 0x7f1cdc0ce0b0 con 0x7f1cdc0aebc0 2023-06-23T00:35:58.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.480+0000 7f1cdbfff700 1 --2- 172.21.15.55:0/2922612390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b41d0 0x7f1cdc0b0760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00098e0 tx=0x7f1cd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3819e637784baf63 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:58.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.480+0000 7f1cdb7fe700 1 -- 172.21.15.55:0/2922612390 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0009e80 con 0x7f1cdc0b41d0 2023-06-23T00:35:58.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.480+0000 7f1cdb7fe700 1 -- 172.21.15.55:0/2922612390 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd000b8f0 con 0x7f1cdc0b41d0 2023-06-23T00:35:58.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.480+0000 7f1cdb7fe700 1 -- 172.21.15.55:0/2922612390 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000bc70 con 0x7f1cdc0b41d0 2023-06-23T00:35:58.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.481+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2922612390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b41d0 msgr2=0x7f1cdc0b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.481+0000 7f1ce2e6a700 1 --2- 172.21.15.55:0/2922612390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b41d0 0x7f1cdc0b0760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00098e0 tx=0x7f1cd00091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.481+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2922612390 shutdown_connections 2023-06-23T00:35:58.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.481+0000 7f1ce2e6a700 1 --2- 172.21.15.55:0/2922612390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b41d0 0x7f1cdc0b0760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.481+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2922612390 >> 172.21.15.55:0/2922612390 conn(0x7f1cdc067bf0 msgr2=0x7f1cdc0ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:58.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.481+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2922612390 shutdown_connections 2023-06-23T00:35:58.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.481+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2922612390 wait complete. 2023-06-23T00:35:58.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.482+0000 7f1ce2e6a700 1 Processor -- start 2023-06-23T00:35:58.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.482+0000 7f1ce2e6a700 1 -- start start 2023-06-23T00:35:58.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce2e6a700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1cdc0aebc0 0x7f1cdc14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:58.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce2e6a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b3e00 0x7f1cdc0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:58.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce2e6a700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1cdc0b41d0 0x7f1cdc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:58.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce2e6a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0c5350 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce2e6a700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0c7670 con 0x7f1cdc0b41d0 2023-06-23T00:35:58.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce2e6a700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0caa40 con 0x7f1cdc0aebc0 2023-06-23T00:35:58.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cdbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b3e00 0x7f1cdc0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:58.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cdbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b3e00 0x7f1cdc0c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39610/0 (socket says 172.21.15.55:39610) 2023-06-23T00:35:58.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cdbfff700 1 -- 172.21.15.55:0/2856862232 learned_addr learned my addr 172.21.15.55:0/2856862232 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:58.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce1407700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1cdc0b41d0 0x7f1cdc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:58.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce0c06700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1cdc0aebc0 0x7f1cdc14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:58.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cdbfff700 1 -- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1cdc0aebc0 msgr2=0x7f1cdc14ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cdbfff700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1cdc0aebc0 0x7f1cdc14ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cdbfff700 1 -- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1cdc0b41d0 msgr2=0x7f1cdc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cdbfff700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1cdc0b41d0 0x7f1cdc0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cdbfff700 1 -- 172.21.15.55:0/2856862232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0009420 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce1407700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1cdc0b41d0 0x7f1cdc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T00:35:58.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cdbfff700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b3e00 0x7f1cdc0c9e80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0013000 tx=0x7f1cd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:58.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cd8ff9700 1 -- 172.21.15.55:0/2856862232 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0024440 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cd8ff9700 1 -- 172.21.15.55:0/2856862232 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd00245a0 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2856862232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc0c4a20 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1cd8ff9700 1 -- 172.21.15.55:0/2856862232 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0024960 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2856862232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc0c4fe0 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.484+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2856862232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc0c54c0 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.485+0000 7f1cd8ff9700 1 -- 172.21.15.55:0/2856862232 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f1cd001a630 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.485+0000 7f1cd8ff9700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1cc8066e90 0x7f1cc8069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:58.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.486+0000 7f1cd8ff9700 1 -- 172.21.15.55:0/2856862232 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f1cd00866e0 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.486+0000 7f1ce0c06700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1cc8066e90 0x7f1cc8069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:58.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.486+0000 7f1ce0c06700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1cc8066e90 0x7f1cc8069340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e230 tx=0x7f1cd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:58.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.491+0000 7f1cd8ff9700 1 -- 172.21.15.55:0/2856862232 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f1cd0053910 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.731+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2856862232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1cdc04c1b0 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.738+0000 7f1cd8ff9700 1 -- 172.21.15.55:0/2856862232 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1cd0056f30 con 0x7f1cdc0b3e00 2023-06-23T00:35:58.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.743+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1cc8066e90 msgr2=0x7f1cc8069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.743+0000 7f1ce2e6a700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1cc8066e90 0x7f1cc8069340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e230 tx=0x7f1cd4009040 comp rx=0 tx=0).stop 2023-06-23T00:35:58.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.743+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2856862232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b3e00 msgr2=0x7f1cdc0c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.743+0000 7f1ce2e6a700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b3e00 0x7f1cdc0c9e80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0013000 tx=0x7f1cd0017000 comp rx=0 tx=0).stop 2023-06-23T00:35:58.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.752+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2856862232 shutdown_connections 2023-06-23T00:35:58.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.752+0000 7f1ce2e6a700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f1cc8066e90 0x7f1cc8069340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.752+0000 7f1ce2e6a700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f1cdc0b41d0 0x7f1cdc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.752+0000 7f1ce2e6a700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f1cdc0aebc0 0x7f1cdc14ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.752+0000 7f1ce2e6a700 1 --2- 172.21.15.55:0/2856862232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cdc0b3e00 0x7f1cdc0c9e80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.752+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2856862232 >> 172.21.15.55:0/2856862232 conn(0x7f1cdc067bf0 msgr2=0x7f1cdc159310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:58.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.756+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2856862232 shutdown_connections 2023-06-23T00:35:58.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.757+0000 7f1ce2e6a700 1 -- 172.21.15.55:0/2856862232 wait complete. 2023-06-23T00:35:58.768 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 3 and data pool 4 2023-06-23T00:35:58.778 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T00:35:58.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.894+0000 7f8505905700 1 Processor -- start 2023-06-23T00:35:58.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.894+0000 7f8505905700 1 -- start start 2023-06-23T00:35:58.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.895+0000 7f8505905700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f85000ad2d0 0x7f8500164030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:58.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.895+0000 7f8505905700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850006a600 con 0x7f8500159190 2023-06-23T00:35:58.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.895+0000 7f8505905700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85000af6c0 con 0x7f8500068100 2023-06-23T00:35:58.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.895+0000 7f8505905700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85000af9a0 con 0x7f85000ad2d0 2023-06-23T00:35:58.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.895+0000 7f8504903700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8500159190 0x7f85000aec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56962/0 (socket says 172.21.15.55:56962) 2023-06-23T00:35:58.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.895+0000 7f8504903700 1 -- 172.21.15.55:0/4133768983 learned_addr learned my addr 172.21.15.55:0/4133768983 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:58.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.895+0000 7f8505104700 1 --2- 172.21.15.55:0/4133768983 >> v2:172.21.15.55:3300/0 conn(0x7f85000ad2d0 0x7f8500164030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:58.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.896+0000 7f84ff7fe700 1 -- 172.21.15.55:0/4133768983 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225465868 0 0) 0x7f850006a600 con 0x7f8500159190 2023-06-23T00:35:58.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.896+0000 7f84ff7fe700 1 -- 172.21.15.55:0/4133768983 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84e8003570 con 0x7f8500159190 2023-06-23T00:35:58.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.896+0000 7f8505104700 1 -- 172.21.15.55:0/4133768983 >> v1:172.21.15.141:6789/0 conn(0x7f8500068100 legacy=0x7f85000acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:58.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.896+0000 7f8505104700 1 -- 172.21.15.55:0/4133768983 >> v1:172.21.15.55:6789/0 conn(0x7f8500159190 legacy=0x7f85000aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.896+0000 7f8505104700 1 -- 172.21.15.55:0/4133768983 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8500069fb0 con 0x7f85000ad2d0 2023-06-23T00:35:58.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.896+0000 7f8505104700 1 --2- 172.21.15.55:0/4133768983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85000ad2d0 0x7f8500164030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009910 tx=0x7f84f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12df39a32f056720 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:58.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.896+0000 7f84ff7fe700 1 -- 172.21.15.55:0/4133768983 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0011630 con 0x7f85000ad2d0 2023-06-23T00:35:58.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.896+0000 7f84ff7fe700 1 -- 172.21.15.55:0/4133768983 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f0011790 con 0x7f85000ad2d0 2023-06-23T00:35:58.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.896+0000 7f84ff7fe700 1 -- 172.21.15.55:0/4133768983 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0011b50 con 0x7f85000ad2d0 2023-06-23T00:35:58.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.897+0000 7f8505905700 1 -- 172.21.15.55:0/4133768983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85000ad2d0 msgr2=0x7f8500164030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.897+0000 7f8505905700 1 --2- 172.21.15.55:0/4133768983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85000ad2d0 0x7f8500164030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009910 tx=0x7f84f00091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.897+0000 7f8505905700 1 -- 172.21.15.55:0/4133768983 shutdown_connections 2023-06-23T00:35:58.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.897+0000 7f8505905700 1 --2- 172.21.15.55:0/4133768983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85000ad2d0 0x7f8500164030 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.897+0000 7f8505905700 1 -- 172.21.15.55:0/4133768983 >> 172.21.15.55:0/4133768983 conn(0x7f85000cdbe0 msgr2=0x7f85000abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:58.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.897+0000 7f8505905700 1 -- 172.21.15.55:0/4133768983 shutdown_connections 2023-06-23T00:35:58.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.897+0000 7f8505905700 1 -- 172.21.15.55:0/4133768983 wait complete. 2023-06-23T00:35:58.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.898+0000 7f8505905700 1 Processor -- start 2023-06-23T00:35:58.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.898+0000 7f8505905700 1 -- start start 2023-06-23T00:35:58.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8505905700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8500068100 0x7f8500152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:58.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8505905700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f85000ad2d0 0x7f850015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:58.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8505905700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8500159190 0x7f850006e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:58.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8505905700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8500153240 con 0x7f8500068100 2023-06-23T00:35:58.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8505905700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850015d8b0 con 0x7f8500159190 2023-06-23T00:35:58.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8505905700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850015db90 con 0x7f85000ad2d0 2023-06-23T00:35:58.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8504903700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8500068100 0x7f8500152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:58.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8504903700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8500068100 0x7f8500152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39624/0 (socket says 172.21.15.55:39624) 2023-06-23T00:35:58.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8504903700 1 -- 172.21.15.55:0/1234203096 learned_addr learned my addr 172.21.15.55:0/1234203096 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:58.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8505104700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8500159190 0x7f850006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:58.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f84fffff700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f85000ad2d0 0x7f850015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:58.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8504903700 1 -- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f85000ad2d0 msgr2=0x7f850015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8504903700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f85000ad2d0 0x7f850015cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8504903700 1 -- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8500159190 msgr2=0x7f850006e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:58.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8504903700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8500159190 0x7f850006e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:58.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8504903700 1 -- 172.21.15.55:0/1234203096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0009420 con 0x7f8500068100 2023-06-23T00:35:58.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8505104700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8500159190 0x7f850006e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T00:35:58.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8504903700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8500068100 0x7f8500152d00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f84f800cc40 tx=0x7f84f8001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:58.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f84fcff9700 1 -- 172.21.15.55:0/1234203096 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f800a9c0 con 0x7f8500068100 2023-06-23T00:35:58.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f84fcff9700 1 -- 172.21.15.55:0/1234203096 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f800ab40 con 0x7f8500068100 2023-06-23T00:35:58.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f8505905700 1 -- 172.21.15.55:0/1234203096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850015de70 con 0x7f8500068100 2023-06-23T00:35:58.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.899+0000 7f84fcff9700 1 -- 172.21.15.55:0/1234203096 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f80070f0 con 0x7f8500068100 2023-06-23T00:35:58.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.900+0000 7f8505905700 1 -- 172.21.15.55:0/1234203096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850015e430 con 0x7f8500068100 2023-06-23T00:35:58.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.901+0000 7f8505905700 1 -- 172.21.15.55:0/1234203096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850006d210 con 0x7f8500068100 2023-06-23T00:35:58.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.901+0000 7f84fcff9700 1 -- 172.21.15.55:0/1234203096 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f84f8007520 con 0x7f8500068100 2023-06-23T00:35:58.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.901+0000 7f84fcff9700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f84ec066e90 0x7f84ec069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:58.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.902+0000 7f84fcff9700 1 -- 172.21.15.55:0/1234203096 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f84f8084a10 con 0x7f8500068100 2023-06-23T00:35:58.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.902+0000 7f84fffff700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f84ec066e90 0x7f84ec069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:58.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.902+0000 7f84fffff700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f84ec066e90 0x7f84ec069340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85000b0270 tx=0x7f84f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:58.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:58.906+0000 7f84fcff9700 1 -- 172.21.15.55:0/1234203096 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f84f8051c60 con 0x7f8500068100 2023-06-23T00:35:59.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.101+0000 7f8505905700 1 -- 172.21.15.55:0/1234203096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f850015e650 con 0x7f8500068100 2023-06-23T00:35:59.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.102+0000 7f84fcff9700 1 -- 172.21.15.55:0/1234203096 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f84f8055280 con 0x7f8500068100 2023-06-23T00:35:59.107 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:59.108 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":20,"fsid":"c0a83a0b-dd71-486d-bdc4-a203f7ef404f","created":"2023-06-23T00:35:40.890983+0000","modified":"2023-06-23T00:35:58.733860+0000","last_up_change":"2023-06-23T00:35:51.173985+0000","last_in_change":"2023-06-23T00:35:43.351694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T00:35:52.176829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T00:35:52.605332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T00:35:56.424171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T00:35:57.707362+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4590eaf-cc24-4ff1-8f1b-9637138bd300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":87956},{"type":"v1","addr":"172.21.15.55:6817","nonce":87956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":87956},{"type":"v1","addr":"172.21.15.55:6819","nonce":87956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":87956},{"type":"v1","addr":"172.21.15.55:6823","nonce":87956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":87956},{"type":"v1","addr":"172.21.15.55:6821","nonce":87956}]},"public_addr":"172.21.15.55:6817/87956","cluster_addr":"172.21.15.55:6819/87956","heartbeat_back_addr":"172.21.15.55:6823/87956","heartbeat_front_addr":"172.21.15.55:6821/87956","state":["exists","up"]},{"osd":1,"uuid":"518b1bc8-fb1c-405f-8346-fda7978829a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":87959},{"type":"v1","addr":"172.21.15.55:6809","nonce":87959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":87959},{"type":"v1","addr":"172.21.15.55:6811","nonce":87959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":87959},{"type":"v1","addr":"172.21.15.55:6815","nonce":87959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":87959},{"type":"v1","addr":"172.21.15.55:6813","nonce":87959}]},"public_addr":"172.21.15.55:6809/87959","cluster_addr":"172.21.15.55:6811/87959","heartbeat_back_addr":"172.21.15.55:6815/87959","heartbeat_front_addr":"172.21.15.55:6813/87959","state":["exists","up"]},{"osd":2,"uuid":"636a04f1-2379-43ca-9f02-0ee33d2af90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":87955},{"type":"v1","addr":"172.21.15.55:6801","nonce":87955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":87955},{"type":"v1","addr":"172.21.15.55:6803","nonce":87955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":87955},{"type":"v1","addr":"172.21.15.55:6807","nonce":87955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":87955},{"type":"v1","addr":"172.21.15.55:6805","nonce":87955}]},"public_addr":"172.21.15.55:6801/87955","cluster_addr":"172.21.15.55:6803/87955","heartbeat_back_addr":"172.21.15.55:6807/87955","heartbeat_front_addr":"172.21.15.55:6805/87955","state":["exists","up"]},{"osd":3,"uuid":"b893aff1-9f4d-42ca-9af6-42fd6ba3d92c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":87972},{"type":"v1","addr":"172.21.15.55:6825","nonce":87972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":87972},{"type":"v1","addr":"172.21.15.55:6827","nonce":87972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":87972},{"type":"v1","addr":"172.21.15.55:6831","nonce":87972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":87972},{"type":"v1","addr":"172.21.15.55:6829","nonce":87972}]},"public_addr":"172.21.15.55:6825/87972","cluster_addr":"172.21.15.55:6827/87972","heartbeat_back_addr":"172.21.15.55:6831/87972","heartbeat_front_addr":"172.21.15.55:6829/87972","state":["exists","up"]},{"osd":4,"uuid":"259d9777-c081-4c0b-9d83-46d348872e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":88177},{"type":"v1","addr":"172.21.15.141:6817","nonce":88177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":88177},{"type":"v1","addr":"172.21.15.141:6819","nonce":88177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6825","nonce":88177},{"type":"v1","addr":"172.21.15.141:6827","nonce":88177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6821","nonce":88177},{"type":"v1","addr":"172.21.15.141:6823","nonce":88177}]},"public_addr":"172.21.15.141:6817/88177","cluster_addr":"172.21.15.141:6819/88177","heartbeat_back_addr":"172.21.15.141:6827/88177","heartbeat_front_addr":"172.21.15.141:6823/88177","state":["exists","up"]},{"osd":5,"uuid":"56cfcc58-3f4d-4637-ae5d-a2a52dbdc37f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":88161},{"type":"v1","addr":"172.21.15.141:6801","nonce":88161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":88161},{"type":"v1","addr":"172.21.15.141:6803","nonce":88161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":88161},{"type":"v1","addr":"172.21.15.141:6807","nonce":88161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":88161},{"type":"v1","addr":"172.21.15.141:6805","nonce":88161}]},"public_addr":"172.21.15.141:6801/88161","cluster_addr":"172.21.15.141:6803/88161","heartbeat_back_addr":"172.21.15.141:6807/88161","heartbeat_front_addr":"172.21.15.141:6805/88161","state":["exists","up"]},{"osd":6,"uuid":"49f9b91b-16c2-4394-928e-97446f5feea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":88192},{"type":"v1","addr":"172.21.15.141:6822","nonce":88192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":88192},{"type":"v1","addr":"172.21.15.141:6826","nonce":88192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":88192},{"type":"v1","addr":"172.21.15.141:6831","nonce":88192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":88192},{"type":"v1","addr":"172.21.15.141:6829","nonce":88192}]},"public_addr":"172.21.15.141:6822/88192","cluster_addr":"172.21.15.141:6826/88192","heartbeat_back_addr":"172.21.15.141:6831/88192","heartbeat_front_addr":"172.21.15.141:6829/88192","state":["exists","up"]},{"osd":7,"uuid":"e9806e67-d1e0-49c5-a3e4-898bd4f840e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":88162},{"type":"v1","addr":"172.21.15.141:6809","nonce":88162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":88162},{"type":"v1","addr":"172.21.15.141:6811","nonce":88162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":88162},{"type":"v1","addr":"172.21.15.141:6815","nonce":88162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":88162},{"type":"v1","addr":"172.21.15.141:6813","nonce":88162}]},"public_addr":"172.21.15.141:6809/88162","cluster_addr":"172.21.15.141:6811/88162","heartbeat_back_addr":"172.21.15.141:6815/88162","heartbeat_front_addr":"172.21.15.141:6813/88162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.504877+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.470542+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.458869+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.499431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.550397+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.286978+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.541488+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.226174+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T00:35:59.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.104+0000 7f8505905700 1 -- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f84ec066e90 msgr2=0x7f84ec069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.104+0000 7f8505905700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f84ec066e90 0x7f84ec069340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85000b0270 tx=0x7f84f4009040 comp rx=0 tx=0).stop 2023-06-23T00:35:59.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.104+0000 7f8505905700 1 -- 172.21.15.55:0/1234203096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8500068100 msgr2=0x7f8500152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.104+0000 7f8505905700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8500068100 0x7f8500152d00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f84f800cc40 tx=0x7f84f8001a70 comp rx=0 tx=0).stop 2023-06-23T00:35:59.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.104+0000 7f8505905700 1 -- 172.21.15.55:0/1234203096 shutdown_connections 2023-06-23T00:35:59.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.104+0000 7f8505905700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f84ec066e90 0x7f84ec069340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.104+0000 7f8505905700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f8500159190 0x7f850006e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.104+0000 7f8505905700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8500068100 0x7f8500152d00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.104+0000 7f8505905700 1 --2- 172.21.15.55:0/1234203096 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f85000ad2d0 0x7f850015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.104+0000 7f8505905700 1 -- 172.21.15.55:0/1234203096 >> 172.21.15.55:0/1234203096 conn(0x7f85000cdbe0 msgr2=0x7f85000aa340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:59.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.105+0000 7f8505905700 1 -- 172.21.15.55:0/1234203096 shutdown_connections 2023-06-23T00:35:59.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.105+0000 7f8505905700 1 -- 172.21.15.55:0/1234203096 wait complete. 2023-06-23T00:35:59.120 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T00:35:59.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.239+0000 7fbec4a16700 1 Processor -- start 2023-06-23T00:35:59.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.239+0000 7fbec4a16700 1 -- start start 2023-06-23T00:35:59.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.240+0000 7fbec4a16700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7fbec00b24e0 0x7fbec00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.240+0000 7fbec4a16700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec00cec20 con 0x7fbec0142de0 2023-06-23T00:35:59.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.240+0000 7fbec4a16700 1 -- --> v1:172.21.15.141:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec00b5ae0 con 0x7fbec00b52f0 2023-06-23T00:35:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.240+0000 7fbec4a16700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec00b5dc0 con 0x7fbec00b24e0 2023-06-23T00:35:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.240+0000 7fbebf7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbec0142de0 0x7fbec00b1dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56966/0 (socket says 172.21.15.55:56966) 2023-06-23T00:35:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.240+0000 7fbebf7fe700 1 -- 172.21.15.55:0/1165763637 learned_addr learned my addr 172.21.15.55:0/1165763637 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.240+0000 7fbebeffd700 1 --2- 172.21.15.55:0/1165763637 >> v2:172.21.15.141:3300/0 conn(0x7fbec00b24e0 0x7fbec00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.240+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482196612 0 0) 0x7fbec00cec20 con 0x7fbec0142de0 2023-06-23T00:35:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.240+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea8003570 con 0x7fbec0142de0 2023-06-23T00:35:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1900104178 0 0) 0x7fbea8003570 con 0x7fbec0142de0 2023-06-23T00:35:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 >> v2:172.21.15.141:3300/0 conn(0x7fbec00b24e0 msgr2=0x7fbec00b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 --2- 172.21.15.55:0/1165763637 >> v2:172.21.15.141:3300/0 conn(0x7fbec00b24e0 0x7fbec00b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 >> v1:172.21.15.141:6789/0 conn(0x7fbec00b52f0 legacy=0x7fbec00cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec00b60a0 con 0x7fbec0142de0 2023-06-23T00:35:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2542258539 0 0) 0x7fbeb0001470 con 0x7fbec0142de0 2023-06-23T00:35:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 >> v1:172.21.15.55:6789/0 conn(0x7fbec0142de0 legacy=0x7fbec00b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbea8003a60 0x7fbea8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbea8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fbea80051c0 0x7fbea800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec00b5dc0 con 0x7fbea8004810 2023-06-23T00:35:59.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea800c0d0 con 0x7fbea8003a60 2023-06-23T00:35:59.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea800c290 con 0x7fbea80051c0 2023-06-23T00:35:59.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebeffd700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbea8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebf7fe700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbea8003a60 0x7fbea8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebffff700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fbea80051c0 0x7fbea800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebeffd700 1 -- 172.21.15.55:0/1165763637 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fbea80051c0 msgr2=0x7fbea800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:59.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebeffd700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fbea80051c0 0x7fbea800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebeffd700 1 -- 172.21.15.55:0/1165763637 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbea8003a60 msgr2=0x7fbea8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebeffd700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbea8003a60 0x7fbea8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebffff700 1 -- 172.21.15.55:0/1165763637 reap_dead start 2023-06-23T00:35:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebeffd700 1 -- 172.21.15.55:0/1165763637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea8003570 con 0x7fbea8004810 2023-06-23T00:35:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebeffd700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbea8004c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009c00 tx=0x7fbeb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b8c460c9ea3433 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:59.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.241+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400e710 con 0x7fbea8004810 2023-06-23T00:35:59.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.242+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb400e870 con 0x7fbea8004810 2023-06-23T00:35:59.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.242+0000 7fbebe7fc700 1 -- 172.21.15.55:0/1165763637 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400ec30 con 0x7fbea8004810 2023-06-23T00:35:59.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.242+0000 7fbec4a16700 1 -- 172.21.15.55:0/1165763637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 msgr2=0x7fbea8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.242+0000 7fbec4a16700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbea8004c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009c00 tx=0x7fbeb40091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.242+0000 7fbec4a16700 1 -- 172.21.15.55:0/1165763637 shutdown_connections 2023-06-23T00:35:59.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.242+0000 7fbec4a16700 1 --2- 172.21.15.55:0/1165763637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbea8004c80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.242+0000 7fbec4a16700 1 -- 172.21.15.55:0/1165763637 >> 172.21.15.55:0/1165763637 conn(0x7fbec0070290 msgr2=0x7fbec0070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:59.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.242+0000 7fbec4a16700 1 -- 172.21.15.55:0/1165763637 shutdown_connections 2023-06-23T00:35:59.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.242+0000 7fbec4a16700 1 -- 172.21.15.55:0/1165763637 wait complete. 2023-06-23T00:35:59.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.243+0000 7fbec4a16700 1 Processor -- start 2023-06-23T00:35:59.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.243+0000 7fbec4a16700 1 -- start start 2023-06-23T00:35:59.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbec4a16700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbec0137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbec4a16700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fbec015e1b0 0x7fbec0138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbec4a16700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbec015e600 0x7fbec0138f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbec4a16700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec015b320 con 0x7fbea8004810 2023-06-23T00:35:59.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbec4a16700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec015eb90 con 0x7fbec015e600 2023-06-23T00:35:59.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbec4a16700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec015a9f0 con 0x7fbec015e1b0 2023-06-23T00:35:59.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbebf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbec0137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbebf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbec0137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39642/0 (socket says 172.21.15.55:39642) 2023-06-23T00:35:59.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbebf7fe700 1 -- 172.21.15.55:0/3775994963 learned_addr learned my addr 172.21.15.55:0/3775994963 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:59.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbebffff700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbec015e600 0x7fbec0138f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbebeffd700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fbec015e1b0 0x7fbec0138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbebf7fe700 1 -- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fbec015e1b0 msgr2=0x7fbec0138a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbebf7fe700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fbec015e1b0 0x7fbec0138a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbebf7fe700 1 -- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbec015e600 msgr2=0x7fbec0138f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbebf7fe700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbec015e600 0x7fbec0138f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.244+0000 7fbebf7fe700 1 -- 172.21.15.55:0/3775994963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec015acd0 con 0x7fbea8004810 2023-06-23T00:35:59.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.245+0000 7fbebf7fe700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbec0137ca0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0007390 tx=0x7fbeb0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:59.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.245+0000 7fbea77fe700 1 -- 172.21.15.55:0/3775994963 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0018030 con 0x7fbea8004810 2023-06-23T00:35:59.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.245+0000 7fbea77fe700 1 -- 172.21.15.55:0/3775994963 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb0007770 con 0x7fbea8004810 2023-06-23T00:35:59.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.245+0000 7fbec4a16700 1 -- 172.21.15.55:0/3775994963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec015afb0 con 0x7fbea8004810 2023-06-23T00:35:59.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.245+0000 7fbea77fe700 1 -- 172.21.15.55:0/3775994963 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb000fb20 con 0x7fbea8004810 2023-06-23T00:35:59.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.245+0000 7fbec4a16700 1 -- 172.21.15.55:0/3775994963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec01381e0 con 0x7fbea8004810 2023-06-23T00:35:59.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.246+0000 7fbec4a16700 1 -- 172.21.15.55:0/3775994963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec01383a0 con 0x7fbea8004810 2023-06-23T00:35:59.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.246+0000 7fbea77fe700 1 -- 172.21.15.55:0/3775994963 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fbeb001b070 con 0x7fbea8004810 2023-06-23T00:35:59.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.246+0000 7fbea77fe700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fbeac066e90 0x7fbeac069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.247+0000 7fbea77fe700 1 -- 172.21.15.55:0/3775994963 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbeb0019070 con 0x7fbea8004810 2023-06-23T00:35:59.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.247+0000 7fbebeffd700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fbeac066e90 0x7fbeac069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.247+0000 7fbebeffd700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fbeac066e90 0x7fbeac069340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40091e0 tx=0x7fbeb401a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:59.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.251+0000 7fbea77fe700 1 -- 172.21.15.55:0/3775994963 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fbeb0051520 con 0x7fbea8004810 2023-06-23T00:35:59.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.446+0000 7fbec4a16700 1 -- 172.21.15.55:0/3775994963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbec00ce840 con 0x7fbea8004810 2023-06-23T00:35:59.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.446+0000 7fbea77fe700 1 -- 172.21.15.55:0/3775994963 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16879 (secure 0 0 0) 0x7fbeb0054b40 con 0x7fbea8004810 2023-06-23T00:35:59.450 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-06-23T00:35:59.450 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":20,"fsid":"c0a83a0b-dd71-486d-bdc4-a203f7ef404f","created":"2023-06-23T00:35:40.890983+0000","modified":"2023-06-23T00:35:58.733860+0000","last_up_change":"2023-06-23T00:35:51.173985+0000","last_in_change":"2023-06-23T00:35:43.351694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-23T00:35:52.176829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-23T00:35:52.605332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T00:35:56.424171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T00:35:57.707362+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4590eaf-cc24-4ff1-8f1b-9637138bd300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":87956},{"type":"v1","addr":"172.21.15.55:6817","nonce":87956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":87956},{"type":"v1","addr":"172.21.15.55:6819","nonce":87956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":87956},{"type":"v1","addr":"172.21.15.55:6823","nonce":87956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":87956},{"type":"v1","addr":"172.21.15.55:6821","nonce":87956}]},"public_addr":"172.21.15.55:6817/87956","cluster_addr":"172.21.15.55:6819/87956","heartbeat_back_addr":"172.21.15.55:6823/87956","heartbeat_front_addr":"172.21.15.55:6821/87956","state":["exists","up"]},{"osd":1,"uuid":"518b1bc8-fb1c-405f-8346-fda7978829a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":87959},{"type":"v1","addr":"172.21.15.55:6809","nonce":87959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":87959},{"type":"v1","addr":"172.21.15.55:6811","nonce":87959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":87959},{"type":"v1","addr":"172.21.15.55:6815","nonce":87959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":87959},{"type":"v1","addr":"172.21.15.55:6813","nonce":87959}]},"public_addr":"172.21.15.55:6809/87959","cluster_addr":"172.21.15.55:6811/87959","heartbeat_back_addr":"172.21.15.55:6815/87959","heartbeat_front_addr":"172.21.15.55:6813/87959","state":["exists","up"]},{"osd":2,"uuid":"636a04f1-2379-43ca-9f02-0ee33d2af90a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":87955},{"type":"v1","addr":"172.21.15.55:6801","nonce":87955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":87955},{"type":"v1","addr":"172.21.15.55:6803","nonce":87955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":87955},{"type":"v1","addr":"172.21.15.55:6807","nonce":87955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":87955},{"type":"v1","addr":"172.21.15.55:6805","nonce":87955}]},"public_addr":"172.21.15.55:6801/87955","cluster_addr":"172.21.15.55:6803/87955","heartbeat_back_addr":"172.21.15.55:6807/87955","heartbeat_front_addr":"172.21.15.55:6805/87955","state":["exists","up"]},{"osd":3,"uuid":"b893aff1-9f4d-42ca-9af6-42fd6ba3d92c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":87972},{"type":"v1","addr":"172.21.15.55:6825","nonce":87972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":87972},{"type":"v1","addr":"172.21.15.55:6827","nonce":87972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":87972},{"type":"v1","addr":"172.21.15.55:6831","nonce":87972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":87972},{"type":"v1","addr":"172.21.15.55:6829","nonce":87972}]},"public_addr":"172.21.15.55:6825/87972","cluster_addr":"172.21.15.55:6827/87972","heartbeat_back_addr":"172.21.15.55:6831/87972","heartbeat_front_addr":"172.21.15.55:6829/87972","state":["exists","up"]},{"osd":4,"uuid":"259d9777-c081-4c0b-9d83-46d348872e01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6816","nonce":88177},{"type":"v1","addr":"172.21.15.141:6817","nonce":88177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6818","nonce":88177},{"type":"v1","addr":"172.21.15.141:6819","nonce":88177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6825","nonce":88177},{"type":"v1","addr":"172.21.15.141:6827","nonce":88177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6821","nonce":88177},{"type":"v1","addr":"172.21.15.141:6823","nonce":88177}]},"public_addr":"172.21.15.141:6817/88177","cluster_addr":"172.21.15.141:6819/88177","heartbeat_back_addr":"172.21.15.141:6827/88177","heartbeat_front_addr":"172.21.15.141:6823/88177","state":["exists","up"]},{"osd":5,"uuid":"56cfcc58-3f4d-4637-ae5d-a2a52dbdc37f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6800","nonce":88161},{"type":"v1","addr":"172.21.15.141:6801","nonce":88161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6802","nonce":88161},{"type":"v1","addr":"172.21.15.141:6803","nonce":88161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6806","nonce":88161},{"type":"v1","addr":"172.21.15.141:6807","nonce":88161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6804","nonce":88161},{"type":"v1","addr":"172.21.15.141:6805","nonce":88161}]},"public_addr":"172.21.15.141:6801/88161","cluster_addr":"172.21.15.141:6803/88161","heartbeat_back_addr":"172.21.15.141:6807/88161","heartbeat_front_addr":"172.21.15.141:6805/88161","state":["exists","up"]},{"osd":6,"uuid":"49f9b91b-16c2-4394-928e-97446f5feea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6820","nonce":88192},{"type":"v1","addr":"172.21.15.141:6822","nonce":88192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6824","nonce":88192},{"type":"v1","addr":"172.21.15.141:6826","nonce":88192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6830","nonce":88192},{"type":"v1","addr":"172.21.15.141:6831","nonce":88192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6828","nonce":88192},{"type":"v1","addr":"172.21.15.141:6829","nonce":88192}]},"public_addr":"172.21.15.141:6822/88192","cluster_addr":"172.21.15.141:6826/88192","heartbeat_back_addr":"172.21.15.141:6831/88192","heartbeat_front_addr":"172.21.15.141:6829/88192","state":["exists","up"]},{"osd":7,"uuid":"e9806e67-d1e0-49c5-a3e4-898bd4f840e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6808","nonce":88162},{"type":"v1","addr":"172.21.15.141:6809","nonce":88162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6810","nonce":88162},{"type":"v1","addr":"172.21.15.141:6811","nonce":88162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6814","nonce":88162},{"type":"v1","addr":"172.21.15.141:6815","nonce":88162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.141:6812","nonce":88162},{"type":"v1","addr":"172.21.15.141:6813","nonce":88162}]},"public_addr":"172.21.15.141:6809/88162","cluster_addr":"172.21.15.141:6811/88162","heartbeat_back_addr":"172.21.15.141:6815/88162","heartbeat_front_addr":"172.21.15.141:6813/88162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.504877+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.470542+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.458869+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.499431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.550397+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.286978+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.541488+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T00:35:48.226174+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T00:35:59.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 -- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fbeac066e90 msgr2=0x7fbeac069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fbeac066e90 0x7fbeac069340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40091e0 tx=0x7fbeb401a000 comp rx=0 tx=0).stop 2023-06-23T00:35:59.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 -- 172.21.15.55:0/3775994963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 msgr2=0x7fbec0137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbec0137ca0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0007390 tx=0x7fbeb0001e50 comp rx=0 tx=0).stop 2023-06-23T00:35:59.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 -- 172.21.15.55:0/3775994963 shutdown_connections 2023-06-23T00:35:59.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7fbeac066e90 0x7fbeac069340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7fbec015e600 0x7fbec0138f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbea8004810 0x7fbec0137ca0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 --2- 172.21.15.55:0/3775994963 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7fbec015e1b0 0x7fbec0138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 -- 172.21.15.55:0/3775994963 >> 172.21.15.55:0/3775994963 conn(0x7fbec0070290 msgr2=0x7fbec0141f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:59.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 -- 172.21.15.55:0/3775994963 shutdown_connections 2023-06-23T00:35:59.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.449+0000 7fbec4a16700 1 -- 172.21.15.55:0/3775994963 wait complete. 2023-06-23T00:35:59.465 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-23T00:35:59.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.584+0000 7f7a4068d700 1 Processor -- start 2023-06-23T00:35:59.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.584+0000 7f7a4068d700 1 -- start start 2023-06-23T00:35:59.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.585+0000 7f7a4068d700 1 --2- >> v2:172.21.15.141:3300/0 conn(0x7f7a38153a80 0x7f7a380d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.585+0000 7f7a4068d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a380b2010 0x7f7a380b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.585+0000 7f7a4068d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a380cedd0 con 0x7f7a38153e50 2023-06-23T00:35:59.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.585+0000 7f7a4068d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a380d14c0 con 0x7f7a380b2010 2023-06-23T00:35:59.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.585+0000 7f7a4068d700 1 -- --> v2:172.21.15.141:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a380b29d0 con 0x7f7a38153a80 2023-06-23T00:35:59.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.585+0000 7f7a3ec2a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a380b2010 0x7f7a380b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.585+0000 7f7a3dc28700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7a38153e50 0x7f7a3815a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56976/0 (socket says 172.21.15.55:56976) 2023-06-23T00:35:59.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.585+0000 7f7a3dc28700 1 -- 172.21.15.55:0/2324446696 learned_addr learned my addr 172.21.15.55:0/2324446696 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:59.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a3e429700 1 --2- 172.21.15.55:0/2324446696 >> v2:172.21.15.141:3300/0 conn(0x7f7a38153a80 0x7f7a380d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a3ec2a700 1 -- 172.21.15.55:0/2324446696 >> v2:172.21.15.141:3300/0 conn(0x7f7a38153a80 msgr2=0x7f7a380d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:59.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a3ec2a700 1 --2- 172.21.15.55:0/2324446696 >> v2:172.21.15.141:3300/0 conn(0x7f7a38153a80 0x7f7a380d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a3ec2a700 1 -- 172.21.15.55:0/2324446696 >> v1:172.21.15.55:6789/0 conn(0x7f7a38153e50 legacy=0x7f7a3815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a3ec2a700 1 -- 172.21.15.55:0/2324446696 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3815aed0 con 0x7f7a380b2010 2023-06-23T00:35:59.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a3ec2a700 1 --2- 172.21.15.55:0/2324446696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a380b2010 0x7f7a380b2490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7a380b3100 tx=0x7f7a300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8744b9754891f4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:59.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a3d427700 1 -- 172.21.15.55:0/2324446696 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30009e80 con 0x7f7a380b2010 2023-06-23T00:35:59.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a3d427700 1 -- 172.21.15.55:0/2324446696 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3000b820 con 0x7f7a380b2010 2023-06-23T00:35:59.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a3d427700 1 -- 172.21.15.55:0/2324446696 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000bba0 con 0x7f7a380b2010 2023-06-23T00:35:59.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a4068d700 1 -- 172.21.15.55:0/2324446696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a380b2010 msgr2=0x7f7a380b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.586+0000 7f7a4068d700 1 --2- 172.21.15.55:0/2324446696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a380b2010 0x7f7a380b2490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7a380b3100 tx=0x7f7a300091a0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.587+0000 7f7a4068d700 1 -- 172.21.15.55:0/2324446696 shutdown_connections 2023-06-23T00:35:59.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.587+0000 7f7a4068d700 1 --2- 172.21.15.55:0/2324446696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a380b2010 0x7f7a380b2490 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.587+0000 7f7a4068d700 1 --2- 172.21.15.55:0/2324446696 >> v2:172.21.15.141:3300/0 conn(0x7f7a38153a80 0x7f7a380d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.587+0000 7f7a4068d700 1 -- 172.21.15.55:0/2324446696 >> 172.21.15.55:0/2324446696 conn(0x7f7a38070290 msgr2=0x7f7a38070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:59.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.587+0000 7f7a4068d700 1 -- 172.21.15.55:0/2324446696 shutdown_connections 2023-06-23T00:35:59.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.587+0000 7f7a4068d700 1 -- 172.21.15.55:0/2324446696 wait complete. 2023-06-23T00:35:59.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.588+0000 7f7a4068d700 1 Processor -- start 2023-06-23T00:35:59.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.588+0000 7f7a4068d700 1 -- start start 2023-06-23T00:35:59.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.588+0000 7f7a4068d700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7a380b2010 0x7f7a38145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.588+0000 7f7a4068d700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f7a38153a80 0x7f7a380cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.588+0000 7f7a4068d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a38153e50 0x7f7a380d0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.588+0000 7f7a4068d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38156040 con 0x7f7a38153e50 2023-06-23T00:35:59.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.588+0000 7f7a4068d700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38155430 con 0x7f7a380b2010 2023-06-23T00:35:59.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.588+0000 7f7a4068d700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38155710 con 0x7f7a38153a80 2023-06-23T00:35:59.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3ec2a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a38153e50 0x7f7a380d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3ec2a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a38153e50 0x7f7a380d0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39654/0 (socket says 172.21.15.55:39654) 2023-06-23T00:35:59.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3ec2a700 1 -- 172.21.15.55:0/22320245 learned_addr learned my addr 172.21.15.55:0/22320245 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3e429700 1 --2- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7a380b2010 0x7f7a38145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3dc28700 1 --2- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f7a38153a80 0x7f7a380cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3ec2a700 1 -- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f7a38153a80 msgr2=0x7f7a380cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3ec2a700 1 --2- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f7a38153a80 0x7f7a380cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3ec2a700 1 -- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7a380b2010 msgr2=0x7f7a38145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3ec2a700 1 --2- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7a380b2010 0x7f7a38145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3ec2a700 1 -- 172.21.15.55:0/22320245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a30009420 con 0x7f7a38153e50 2023-06-23T00:35:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3e429700 1 --2- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f7a380b2010 0x7f7a38145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T00:35:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3dc28700 1 --2- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f7a38153a80 0x7f7a380cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T00:35:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a3ec2a700 1 --2- 172.21.15.55:0/22320245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a38153e50 0x7f7a380d0140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a38089000 tx=0x7f7a30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a367fc700 1 -- 172.21.15.55:0/22320245 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3001f070 con 0x7f7a38153e50 2023-06-23T00:35:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a367fc700 1 -- 172.21.15.55:0/22320245 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a30011420 con 0x7f7a38153e50 2023-06-23T00:35:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a4068d700 1 -- 172.21.15.55:0/22320245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a381559f0 con 0x7f7a38153e50 2023-06-23T00:35:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a367fc700 1 -- 172.21.15.55:0/22320245 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000bba0 con 0x7f7a38153e50 2023-06-23T00:35:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.589+0000 7f7a4068d700 1 -- 172.21.15.55:0/22320245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a38146440 con 0x7f7a38153e50 2023-06-23T00:35:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.590+0000 7f7a4068d700 1 -- 172.21.15.55:0/22320245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a381561b0 con 0x7f7a38153e50 2023-06-23T00:35:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.591+0000 7f7a367fc700 1 -- 172.21.15.55:0/22320245 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f7a30011ad0 con 0x7f7a38153e50 2023-06-23T00:35:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.591+0000 7f7a367fc700 1 --2- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7a24066e90 0x7f7a24069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.591+0000 7f7a367fc700 1 -- 172.21.15.55:0/22320245 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7a30085bd0 con 0x7f7a38153e50 2023-06-23T00:35:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.591+0000 7f7a3e429700 1 --2- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7a24066e90 0x7f7a24069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.592+0000 7f7a3e429700 1 --2- 172.21.15.55:0/22320245 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f7a24066e90 0x7f7a24069340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a3815bd90 tx=0x7f7a2800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:59.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.595+0000 7f7a367fc700 1 -- 172.21.15.55:0/22320245 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f7a30052da0 con 0x7f7a38153e50 2023-06-23T00:35:59.841 INFO:teuthology.orchestra.run.smithi055.stderr:no valid command found; 1 closest matches: 2023-06-23T00:35:59.841 INFO:teuthology.orchestra.run.smithi055.stderr:fs set |refuse_client_session|max_xattr_size [--yes-i-really-mean-it] [--yes-i-really-really-mean-it] 2023-06-23T00:35:59.841 INFO:teuthology.orchestra.run.smithi055.stderr:Error EINVAL: invalid command 2023-06-23T00:35:59.844 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-06-23T00:35:59.845 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-23T00:35:59.846 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-23T00:35:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.966+0000 7f097188e700 1 Processor -- start 2023-06-23T00:35:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.966+0000 7f097188e700 1 -- start start 2023-06-23T00:35:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097188e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f096c159310 0x7f096c1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097188e700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f096c0b23d0 0x7f096c15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097188e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096c0cb7d0 con 0x7f096c0b00e0 2023-06-23T00:35:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097188e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096c0b2e10 con 0x7f096c159310 2023-06-23T00:35:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097188e700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0b15c0 con 0x7f096c0b23d0 2023-06-23T00:35:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097088c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f096c159310 0x7f096c1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097108d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f096c0b00e0 0x7f096c0683a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56990/0 (socket says 172.21.15.55:56990) 2023-06-23T00:35:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097108d700 1 -- 172.21.15.55:0/3018784241 learned_addr learned my addr 172.21.15.55:0/3018784241 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-06-23T00:35:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f096bfff700 1 --2- 172.21.15.55:0/3018784241 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f096c0b23d0 0x7f096c15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097088c700 1 -- 172.21.15.55:0/3018784241 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f096c0b23d0 msgr2=0x7f096c15dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T00:35:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097088c700 1 --2- 172.21.15.55:0/3018784241 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f096c0b23d0 0x7f096c15dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097088c700 1 -- 172.21.15.55:0/3018784241 >> v1:172.21.15.55:6789/0 conn(0x7f096c0b00e0 legacy=0x7f096c0683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f097088c700 1 -- 172.21.15.55:0/3018784241 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c15e150 con 0x7f096c159310 2023-06-23T00:35:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.967+0000 7f096b7fe700 1 -- 172.21.15.55:0/3018784241 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890553889 0 0) 0x7f096c0cb7d0 con 0x7f096c0b00e0 2023-06-23T00:35:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f097088c700 1 --2- 172.21.15.55:0/3018784241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f096c159310 0x7f096c1596e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0964009e60 tx=0x7f0964009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665574604f4274f6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f096b7fe700 1 -- 172.21.15.55:0/3018784241 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096400e070 con 0x7f096c159310 2023-06-23T00:35:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f096b7fe700 1 -- 172.21.15.55:0/3018784241 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096400bc30 con 0x7f096c159310 2023-06-23T00:35:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f096b7fe700 1 -- 172.21.15.55:0/3018784241 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0964012430 con 0x7f096c159310 2023-06-23T00:35:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f097188e700 1 -- 172.21.15.55:0/3018784241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f096c159310 msgr2=0x7f096c1596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f097188e700 1 --2- 172.21.15.55:0/3018784241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f096c159310 0x7f096c1596e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0964009e60 tx=0x7f0964009780 comp rx=0 tx=0).stop 2023-06-23T00:35:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f097188e700 1 -- 172.21.15.55:0/3018784241 shutdown_connections 2023-06-23T00:35:59.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f097188e700 1 --2- 172.21.15.55:0/3018784241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f096c159310 0x7f096c1596e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f097188e700 1 --2- 172.21.15.55:0/3018784241 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f096c0b23d0 0x7f096c15dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f097188e700 1 -- 172.21.15.55:0/3018784241 >> 172.21.15.55:0/3018784241 conn(0x7f096c070290 msgr2=0x7f096c070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T00:35:59.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f097188e700 1 -- 172.21.15.55:0/3018784241 shutdown_connections 2023-06-23T00:35:59.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.968+0000 7f097188e700 1 -- 172.21.15.55:0/3018784241 wait complete. 2023-06-23T00:35:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.969+0000 7f097188e700 1 Processor -- start 2023-06-23T00:35:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.970+0000 7f097188e700 1 -- start start 2023-06-23T00:35:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.970+0000 7f097188e700 1 --2- >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f096c0b00e0 0x7f096c0c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.970+0000 7f097188e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f096c159310 0x7f096c0c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.970+0000 7f097188e700 1 --2- >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f096c146ba0 0x7f096c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.970+0000 7f097188e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0c51f0 con 0x7f096c159310 2023-06-23T00:35:59.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.970+0000 7f097188e700 1 -- --> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0c45e0 con 0x7f096c0b00e0 2023-06-23T00:35:59.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.970+0000 7f097188e700 1 -- --> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0c48c0 con 0x7f096c146ba0 2023-06-23T00:35:59.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.970+0000 7f096bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f096c159310 0x7f096c0c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f096bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f096c159310 0x7f096c0c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39668/0 (socket says 172.21.15.55:39668) 2023-06-23T00:35:59.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f096bfff700 1 -- 172.21.15.55:0/1651106945 learned_addr learned my addr 172.21.15.55:0/1651106945 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-06-23T00:35:59.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f097088c700 1 --2- 172.21.15.55:0/1651106945 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f096c0b00e0 0x7f096c0c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f096bfff700 1 -- 172.21.15.55:0/1651106945 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f096c146ba0 msgr2=0x7f096c0c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f097108d700 1 --2- 172.21.15.55:0/1651106945 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f096c146ba0 0x7f096c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f096bfff700 1 --2- 172.21.15.55:0/1651106945 >> [v2:172.21.15.141:3301/0,v1:172.21.15.141:6790/0] conn(0x7f096c146ba0 0x7f096c0c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f096bfff700 1 -- 172.21.15.55:0/1651106945 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f096c0b00e0 msgr2=0x7f096c0c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T00:35:59.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f096bfff700 1 --2- 172.21.15.55:0/1651106945 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f096c0b00e0 0x7f096c0c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T00:35:59.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f096bfff700 1 -- 172.21.15.55:0/1651106945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0964009a20 con 0x7f096c159310 2023-06-23T00:35:59.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f097088c700 1 --2- 172.21.15.55:0/1651106945 >> [v2:172.21.15.141:3300/0,v1:172.21.15.141:6789/0] conn(0x7f096c0b00e0 0x7f096c0c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T00:35:59.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f096bfff700 1 --2- 172.21.15.55:0/1651106945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f096c159310 0x7f096c0c3f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f096000d890 tx=0x7f0960008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:59.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f0968ff9700 1 -- 172.21.15.55:0/1651106945 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600125f0 con 0x7f096c159310 2023-06-23T00:35:59.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f0968ff9700 1 -- 172.21.15.55:0/1651106945 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0960012770 con 0x7f096c159310 2023-06-23T00:35:59.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f097188e700 1 -- 172.21.15.55:0/1651106945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c0c4ba0 con 0x7f096c159310 2023-06-23T00:35:59.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f0968ff9700 1 -- 172.21.15.55:0/1651106945 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960012b30 con 0x7f096c159310 2023-06-23T00:35:59.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.971+0000 7f097188e700 1 -- 172.21.15.55:0/1651106945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c0c93e0 con 0x7f096c159310 2023-06-23T00:35:59.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.972+0000 7f097188e700 1 -- 172.21.15.55:0/1651106945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c0c5040 con 0x7f096c159310 2023-06-23T00:35:59.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.972+0000 7f0968ff9700 1 -- 172.21.15.55:0/1651106945 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f096001e460 con 0x7f096c159310 2023-06-23T00:35:59.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.973+0000 7f0968ff9700 1 --2- 172.21.15.55:0/1651106945 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f0958066e90 0x7f0958069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T00:35:59.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.973+0000 7f0968ff9700 1 -- 172.21.15.55:0/1651106945 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0960084cf0 con 0x7f096c159310 2023-06-23T00:35:59.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.973+0000 7f097088c700 1 --2- 172.21.15.55:0/1651106945 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f0958066e90 0x7f0958069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T00:35:59.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.974+0000 7f097088c700 1 --2- 172.21.15.55:0/1651106945 >> [v2:172.21.15.141:6832/87576,v1:172.21.15.141:6833/87576] conn(0x7f0958066e90 0x7f0958069340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0964009e60 tx=0x7f0964019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T00:35:59.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-06-23T00:35:59.977+0000 7f0968ff9700 1 -- 172.21.15.55:0/1651106945 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f0960089070 con 0x7f096c159310 2023-06-23T00:36:00.223 INFO:teuthology.orchestra.run.smithi055.stderr:no valid command found; 1 closest matches: 2023-06-23T00:36:00.223 INFO:teuthology.orchestra.run.smithi055.stderr:fs set |refuse_client_session|max_xattr_size [--yes-i-really-mean-it] [--yes-i-really-really-mean-it] 2023-06-23T00:36:00.223 INFO:teuthology.orchestra.run.smithi055.stderr:Error EINVAL: invalid command 2023-06-23T00:36:00.226 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-06-23T00:36:00.227 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 450, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 717, in create self.set_allow_standby_replay(standby_replay) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 589, in set_allow_standby_replay self.set_var("allow_standby_replay", yes) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi055 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false' 2023-06-23T00:36:00.227 INFO:teuthology.misc:Shutting down mds daemons... 2023-06-23T00:36:00.228 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-23T00:36:00.229 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.237 INFO:tasks.ceph.mds.a:Stopped 2023-06-23T00:36:00.237 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-23T00:36:00.238 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.289 INFO:tasks.ceph.mds.c:Stopped 2023-06-23T00:36:00.290 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-23T00:36:00.290 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.301 INFO:tasks.ceph.mds.b:Stopped 2023-06-23T00:36:00.301 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-23T00:36:00.302 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.352 INFO:tasks.ceph.mds.d:Stopped 2023-06-23T00:36:00.353 INFO:teuthology.misc:Shutting down osd daemons... 2023-06-23T00:36:00.353 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-06-23T00:36:00.353 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.386 INFO:tasks.ceph.osd.0:Stopped 2023-06-23T00:36:00.386 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-06-23T00:36:00.387 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.452 INFO:tasks.ceph.osd.1:Stopped 2023-06-23T00:36:00.453 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-06-23T00:36:00.453 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.517 INFO:tasks.ceph.osd.2:Stopped 2023-06-23T00:36:00.518 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-06-23T00:36:00.518 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.589 INFO:tasks.ceph.osd.3:Stopped 2023-06-23T00:36:00.589 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-06-23T00:36:00.590 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.636 INFO:tasks.ceph.osd.4:Stopped 2023-06-23T00:36:00.636 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-06-23T00:36:00.637 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.672 INFO:tasks.ceph.osd.5:Stopped 2023-06-23T00:36:00.673 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-06-23T00:36:00.673 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.749 INFO:tasks.ceph.osd.6:Stopped 2023-06-23T00:36:00.750 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-06-23T00:36:00.750 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.824 INFO:tasks.ceph.osd.7:Stopped 2023-06-23T00:36:00.825 INFO:teuthology.misc:Shutting down mgr daemons... 2023-06-23T00:36:00.825 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-06-23T00:36:00.826 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.849 INFO:tasks.ceph.mgr.y:Stopped 2023-06-23T00:36:00.850 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-06-23T00:36:00.850 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.892 INFO:tasks.ceph.mgr.x:Stopped 2023-06-23T00:36:00.892 INFO:teuthology.misc:Shutting down mon daemons... 2023-06-23T00:36:00.893 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-06-23T00:36:00.893 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.904 INFO:tasks.ceph.mon.a:Stopped 2023-06-23T00:36:00.905 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-06-23T00:36:00.905 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.943 INFO:tasks.ceph.mon.b:Stopped 2023-06-23T00:36:00.944 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-06-23T00:36:00.944 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T00:36:00.995 INFO:tasks.ceph.mon.c:Stopped 2023-06-23T00:36:00.996 INFO:tasks.ceph:Checking cluster log for badness... 2023-06-23T00:36:00.996 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-06-23T00:36:01.030 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:36:01.031 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-06-23T00:36:01.430 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:36:01.431 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-06-23T00:36:01.556 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:36:01.557 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-06-23T00:36:01.621 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:36:01.621 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-06-23T00:36:01.712 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:36:01.713 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-06-23T00:36:01.942 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T00:36:01.945 DEBUG:teuthology.orchestra.run.smithi141:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T00:36:02.057 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:36:02.058 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-06-23T00:36:02.169 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:36:02.170 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-06-23T00:36:02.261 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:36:02.262 DEBUG:teuthology.orchestra.run.smithi141:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-06-23T00:36:02.343 INFO:tasks.ceph:Archiving mon data... 2023-06-23T00:36:02.344 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882/data/mon.a.tgz 2023-06-23T00:36:02.345 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2023-06-23T00:36:02.362 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.ewPrjXX1dU 2023-06-23T00:36:02.362 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ewPrjXX1dU 2023-06-23T00:36:02.478 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.ewPrjXX1dU 2023-06-23T00:36:02.562 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.ewPrjXX1dU is 195KB 2023-06-23T00:36:02.613 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.ewPrjXX1dU 2023-06-23T00:36:02.629 DEBUG:teuthology.misc:Transferring archived files from smithi141:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882/data/mon.b.tgz 2023-06-23T00:36:02.629 DEBUG:teuthology.orchestra.run.smithi141:> mktemp 2023-06-23T00:36:02.644 INFO:teuthology.orchestra.run.smithi141.stdout:/tmp/tmp.jTnv7TbD0F 2023-06-23T00:36:02.645 DEBUG:teuthology.orchestra.run.smithi141:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.jTnv7TbD0F 2023-06-23T00:36:02.767 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod 0666 /tmp/tmp.jTnv7TbD0F 2023-06-23T00:36:02.850 DEBUG:teuthology.orchestra.remote:smithi141:/tmp/tmp.jTnv7TbD0F is 270KB 2023-06-23T00:36:02.946 DEBUG:teuthology.orchestra.run.smithi141:> rm -fr /tmp/tmp.jTnv7TbD0F 2023-06-23T00:36:02.961 DEBUG:teuthology.misc:Transferring archived files from smithi141:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882/data/mon.c.tgz 2023-06-23T00:36:02.962 DEBUG:teuthology.orchestra.run.smithi141:> mktemp 2023-06-23T00:36:03.013 INFO:teuthology.orchestra.run.smithi141.stdout:/tmp/tmp.9frTmvWEW5 2023-06-23T00:36:03.014 DEBUG:teuthology.orchestra.run.smithi141:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.9frTmvWEW5 2023-06-23T00:36:03.121 DEBUG:teuthology.orchestra.run.smithi141:> sudo chmod 0666 /tmp/tmp.9frTmvWEW5 2023-06-23T00:36:03.202 DEBUG:teuthology.orchestra.remote:smithi141:/tmp/tmp.9frTmvWEW5 is 270KB 2023-06-23T00:36:03.295 DEBUG:teuthology.orchestra.run.smithi141:> rm -fr /tmp/tmp.9frTmvWEW5 2023-06-23T00:36:03.310 INFO:tasks.ceph:Cleaning ceph cluster... 2023-06-23T00:36:03.310 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-23T00:36:03.313 DEBUG:teuthology.orchestra.run.smithi141:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-23T00:36:03.374 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-06-23T00:36:03.374 DEBUG:teuthology.orchestra.run.smithi055:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-23T00:36:03.380 DEBUG:teuthology.orchestra.run.smithi141:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-23T00:36:03.407 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-23T00:36:03.438 INFO:teuthology.orchestra.run.smithi141.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-23T00:36:03.444 INFO:tasks.ceph:Archiving crash dumps... 2023-06-23T00:36:03.446 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882/remote/smithi055/crash 2023-06-23T00:36:03.447 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-23T00:36:03.480 DEBUG:teuthology.misc:Transferring archived files from smithi141:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882/remote/smithi141/crash 2023-06-23T00:36:03.481 DEBUG:teuthology.orchestra.run.smithi141:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-23T00:36:03.509 INFO:tasks.ceph:Shutting down logrotate 2023-06-23T00:36:03.510 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T00:36:03.525 DEBUG:teuthology.orchestra.run.smithi141:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T00:36:03.604 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-23T00:36:03.631 DEBUG:teuthology.orchestra.run.smithi141:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-23T00:36:03.656 INFO:tasks.ceph:Compressing logs... 2023-06-23T00:36:03.658 DEBUG:teuthology.orchestra.run.smithi055:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T00:36:03.673 DEBUG:teuthology.orchestra.run.smithi141:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T00:36:03.949 INFO:tasks.ceph:Archiving logs... 2023-06-23T00:36:03.950 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882/remote/smithi055/log 2023-06-23T00:36:03.951 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T00:36:04.015 DEBUG:teuthology.misc:Transferring archived files from smithi141:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882/remote/smithi141/log 2023-06-23T00:36:04.016 DEBUG:teuthology.orchestra.run.smithi141:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T00:36:04.074 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 1893, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 450, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 717, in create self.set_allow_standby_replay(standby_replay) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 589, in set_allow_standby_replay self.set_var("allow_standby_replay", yes) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi055 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false' 2023-06-23T00:36:04.149 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=ac61ce5fbf0c4e51a77bb710ad9d936a Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 1893, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 450, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 717, in create self.set_allow_standby_replay(standby_replay) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 589, in set_allow_standby_replay self.set_var("allow_standby_replay", yes) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi055 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false' 2023-06-23T00:36:04.157 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-06-23T00:36:04.170 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-06-23T00:36:04.183 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-06-23T00:36:04.184 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T00:36:04.196 DEBUG:teuthology.orchestra.run.smithi141:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T00:36:04.281 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-06-23T00:36:04.293 INFO:teuthology.task.clock:Checking final clock skew... 2023-06-23T00:36:04.293 DEBUG:teuthology.orchestra.run.smithi055:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T00:36:04.296 DEBUG:teuthology.orchestra.run.smithi141:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T00:36:04.307 INFO:teuthology.orchestra.run.smithi055.stderr:bash: ntpq: command not found 2023-06-23T00:36:04.310 INFO:teuthology.orchestra.run.smithi141.stderr:bash: ntpq: command not found 2023-06-23T00:36:04.311 INFO:teuthology.orchestra.run.smithi055.stdout:210 Number of sources = 4 2023-06-23T00:36:04.312 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T00:36:04.312 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2023-06-23T00:36:04.312 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv01.front.sepia.ceph.com 2 6 77 58 +2288us[+2248us] +/- 114ms 2023-06-23T00:36:04.312 INFO:teuthology.orchestra.run.smithi055.stdout:^* hv02.front.sepia.ceph.com 3 6 77 58 -1547us[-1587us] +/- 65ms 2023-06-23T00:36:04.313 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv03.front.sepia.ceph.com 3 6 77 57 +259us[ +259us] +/- 69ms 2023-06-23T00:36:04.313 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T00:36:04.314 INFO:teuthology.orchestra.run.smithi141.stdout:210 Number of sources = 4 2023-06-23T00:36:04.314 INFO:teuthology.orchestra.run.smithi141.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T00:36:04.314 INFO:teuthology.orchestra.run.smithi141.stdout:=============================================================================== 2023-06-23T00:36:04.314 INFO:teuthology.orchestra.run.smithi141.stdout:^+ hv01.front.sepia.ceph.com 2 6 77 58 +2314us[+2314us] +/- 114ms 2023-06-23T00:36:04.315 INFO:teuthology.orchestra.run.smithi141.stdout:^* hv02.front.sepia.ceph.com 3 6 77 58 -1545us[-1562us] +/- 65ms 2023-06-23T00:36:04.315 INFO:teuthology.orchestra.run.smithi141.stdout:^+ hv03.front.sepia.ceph.com 3 6 77 58 +291us[ +291us] +/- 69ms 2023-06-23T00:36:04.315 INFO:teuthology.orchestra.run.smithi141.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T00:36:04.315 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-06-23T00:36:04.332 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-06-23T00:36:04.333 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-06-23T00:36:04.342 DEBUG:teuthology.orchestra.run.smithi055:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T00:36:04.441 DEBUG:teuthology.orchestra.run.smithi141:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T00:36:04.526 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T00:36:04.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:36:04.557 DEBUG:teuthology.orchestra.run.smithi141:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T00:36:04.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:36:04.587 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-06-23T00:36:04.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-06-23T00:36:04.606 INFO:teuthology.task.internal:Duration was 680.747286 seconds 2023-06-23T00:36:04.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-06-23T00:36:04.615 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-06-23T00:36:04.615 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T00:36:04.618 DEBUG:teuthology.orchestra.run.smithi141:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T00:36:04.661 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T00:36:04.671 INFO:teuthology.orchestra.run.smithi141.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T00:36:04.863 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-06-23T00:36:04.864 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi055.front.sepia.ceph.com 2023-06-23T00:36:04.864 DEBUG:teuthology.orchestra.run.smithi055:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T00:36:04.884 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi141.front.sepia.ceph.com 2023-06-23T00:36:04.884 DEBUG:teuthology.orchestra.run.smithi141:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T00:36:04.904 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-06-23T00:36:04.904 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T00:36:04.927 DEBUG:teuthology.orchestra.run.smithi141:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T00:36:04.981 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-06-23T00:36:04.981 DEBUG:teuthology.orchestra.run.smithi055:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T00:36:05.004 DEBUG:teuthology.orchestra.run.smithi141:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T00:36:05.090 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-06-23T00:36:05.101 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-06-23T00:36:05.102 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T00:36:05.127 DEBUG:teuthology.orchestra.run.smithi141:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T00:36:05.152 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-06-23T00:36:05.162 DEBUG:teuthology.orchestra.run.smithi055:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T00:36:05.169 DEBUG:teuthology.orchestra.run.smithi141:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T00:36:05.192 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = core 2023-06-23T00:36:05.216 INFO:teuthology.orchestra.run.smithi141.stdout:kernel.core_pattern = core 2023-06-23T00:36:05.229 DEBUG:teuthology.orchestra.run.smithi055:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T00:36:05.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:36:05.262 DEBUG:teuthology.orchestra.run.smithi141:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T00:36:05.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:36:05.283 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-06-23T00:36:05.294 INFO:teuthology.task.internal:Transferring archived files... 2023-06-23T00:36:05.295 DEBUG:teuthology.misc:Transferring archived files from smithi055:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882/remote/smithi055 2023-06-23T00:36:05.296 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T00:36:05.335 DEBUG:teuthology.misc:Transferring archived files from smithi141:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882/remote/smithi141 2023-06-23T00:36:05.336 DEBUG:teuthology.orchestra.run.smithi141:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T00:36:05.369 INFO:teuthology.task.internal:Removing archive directory... 2023-06-23T00:36:05.369 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T00:36:05.373 DEBUG:teuthology.orchestra.run.smithi141:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T00:36:05.418 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-06-23T00:36:05.429 INFO:teuthology.task.internal:Not uploading archives. 2023-06-23T00:36:05.430 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-06-23T00:36:05.439 INFO:teuthology.task.internal:Tidying up after the test... 2023-06-23T00:36:05.440 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T00:36:05.442 DEBUG:teuthology.orchestra.run.smithi141:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T00:36:05.454 INFO:teuthology.orchestra.run.smithi055.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 00:36 /home/ubuntu/cephtest 2023-06-23T00:36:05.472 INFO:teuthology.orchestra.run.smithi141.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 00:36 /home/ubuntu/cephtest 2023-06-23T00:36:05.473 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-06-23T00:36:05.486 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-06-23T00:36:05.504 INFO:teuthology.nuke:Checking targets against current locks 2023-06-23T00:36:05.534 DEBUG:teuthology.nuke:shortname: smithi055 2023-06-23T00:36:05.535 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T00:36:05.561 DEBUG:teuthology.nuke:shortname: smithi141 2023-06-23T00:36:05.561 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T00:36:05.565 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:17:06.968133', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC75fSi2oSvKw3YNS36qaAND2urjwcW/v9YT2M3JHk9YXUiOrW0/APthQChdXlMU8j5NDTGTGQX6k2gD31Z7zcB7ZsZ0jxUAgZehlukP0Ss64qMcKhiMdbO/sbQeaHykNmMA1xQlfK91IILupQeAGwQihBbq8ZHEZEc/W9MppBndcjV0Wb/e1e69oAsVmjwd/supjKa6sIEQiCUSuD02zFhfdQIgb+j8GB4mGiQuSwBDT8bhOAtSnaQTXJCKWymD/FdqmSDIUA/dLRdRtxxFg2/u0ANS9350Y/0KWYcgcIPFvCdCAPtW5knw4XhZVnADAWnZ47cV/9MIiTJqYey0zp+t7W8K1Noxa3VkgUdnyKTcD7LTR4XHGFZ1dk4q3LYRV+4cJrjIVtIdC8+3YXaNBL2z0MRhEWtbIQ0vWtiJmOTgrfSKEiGw1blxYOgCiePEKjv956ggIEbQEeq4/DXdq0mJObpfpFc/jKwshNWscFv+gUuGfiEEJpp5kdu0IOdd98='} 2023-06-23T00:36:05.592 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi141.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:17:06.966467', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIXpQjeegzj7I7Qy7Nhs98P522TO+rOlW/zrMBOYH1mMrcIPOJrEA61LILrT0RBuJUpdgtAZhwazYthRMQN5gohIHl4hQy/ahqG+m5nDAP2hZY5sGwad0HgEZB93bjdhPNRBLLKHHpRIIMXZImUsNNwdMpuQxeDtv2/Ts72Qkk/RZ5FpEUkGBIjJ5nfZVtFBF2Sjl1jrZ7eiuoKotuhY1UrO+wcR47RCAQ96xXChs2/h9+vZoag3hcqNGf5oSVZV4QpdWOGmlWPhPrdfkeitw9qVC0UnrKdwvYwuVoO9KdQQYT7JuQIHSei2vw18UeEqubajpjsf+/HePOhzDcNcO5LQc2UU4ouKrntIYH02/3TLUtIFLrvmNXrK6036O/Pg7mVFa9wZ11nelPmbeWCcHME3/8Kki/WyaRKYXZXw+uCCnG/OuCSliYYK49Ap1UvZuku3Qy3jObqADWbT0CMwfxeSX6enw3yPb4hbXmIoC3cd8tt9TMpakPr3x525uNkXc='} 2023-06-23T00:36:05.614 INFO:teuthology.orchestra.console:Power off smithi055 2023-06-23T00:36:05.615 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T00:36:05.641 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T00:36:05.663 INFO:teuthology.orchestra.console:Power off smithi141 2023-06-23T00:36:05.664 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T00:36:05.688 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T00:36:09.676 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T00:36:09.714 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T00:36:13.794 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T00:36:13.826 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T00:36:14.038 INFO:teuthology.orchestra.console:Power off for smithi141 completed 2023-06-23T00:36:17.912 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T00:36:18.125 INFO:teuthology.orchestra.console:Power off for smithi055 completed 2023-06-23T00:36:18.233 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} duration: 680.7472860813141 failure_reason: 'Command failed on smithi055 with status 22: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=ac61ce5fbf0c4e51a77bb710ad9d936a status: fail success: false 2023-06-23T00:36:18.233 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T00:36:18.328 INFO:teuthology.run:FAIL