2023-06-22T16:49:53.366 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-22T16:49:53.367 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814 --owner scheduled_yuriw@teuthology --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} -- /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814/orig.config.yaml 2023-06-22T16:49:53.411 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T16:49:53.501 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814 branch: wip-yuri4-testing-2023-06-21-0846-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} email: yweinste@redhat.com first_in_suite: false job_id: '7311814' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri4-testing-2023-06-21-0846-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2023-06-21-0846-quincy sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9037 sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri4-testing-2023-06-21-0846-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlSCLbZxCowylv4ejcGPoLZwSzztkWsoXph8jwxtsLJ3VXlph3F9wrR1ghn5qjjUtVXhuKxEgDbyX2uINaPFag= smithi202.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDypvRW1ScAmzbCqMdu2b44iXhoAYmmsMtSCXCqMi+j9mOvi4XOcXfKNu/BGvrd49WtXo1PRU4OHW4zMvGMex9eKutRgJWxD9Y3HnFOXkfIaKeFe6odwFQ1RiPLLCSWoeZ5GeIqpqEs9xMEusoo4ly25clw6Ex+7chqlCXWcqt3aVasVUB+46NzjXHQtiPfZn9euC07pub6yE5fowjee3DrA/f8wRRhcXr6EZPuupJkEwQlgwz+dtEA9TG6x/WbPTqyYZ6O4B0Qv9/7IavdhpBLFwH/RbKwnMHDc4drAWjZf+WX65YOMNDvCejLaXT2eyStMp8JisTVu8vXlvUZ7fQjJ29WD3pvzjxHKh5mQdQbYV0m3L2EEBZvLDEidx0YJmlXkZcanoRDbSEeEMHuUP/5nc2OU7OmdEleiydSK1jfVkVyYAbfz7RNpajUaBehbvyxnB5d1Oy1E41DoIEnjIks5Ee4tj6Im2iqIA4bhjyMi8rCjelVj5/8xoJb2AteqYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots - tasks.cephfs.test_volumes.TestSubvolumeSnapshots teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 076bbebc42a14f7d568aaa78eabb0038327bcb23 timestamp: 2023-06-22_15:25:25 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-22T16:49:53.501 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa; will attempt to use it 2023-06-22T16:49:53.502 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks 2023-06-22T16:49:53.502 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-22T16:49:53.504 INFO:teuthology.task.internal:Checking packages... 2023-06-22T16:49:53.532 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '42ddc46e6e6582d8e29a5a982a81886fe028be2c' 2023-06-22T16:49:53.533 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-22T16:49:53.533 INFO:teuthology.packaging:ref: None 2023-06-22T16:49:53.534 INFO:teuthology.packaging:tag: None 2023-06-22T16:49:53.534 INFO:teuthology.packaging:branch: wip-yuri4-testing-2023-06-21-0846-quincy 2023-06-22T16:49:53.534 INFO:teuthology.packaging:sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T16:49:53.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri4-testing-2023-06-21-0846-quincy 2023-06-22T16:49:53.688 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-719.g42ddc46e 2023-06-22T16:49:53.688 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-22T16:49:53.695 INFO:teuthology.task.internal:no buildpackages task found 2023-06-22T16:49:53.695 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-22T16:49:53.738 INFO:teuthology.task.internal:Saving configuration 2023-06-22T16:49:53.754 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-22T16:49:53.768 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T16:49:53.796 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 16:45:04.550447', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlSCLbZxCowylv4ejcGPoLZwSzztkWsoXph8jwxtsLJ3VXlph3F9wrR1ghn5qjjUtVXhuKxEgDbyX2uINaPFag='} 2023-06-22T16:49:53.819 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi202.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 16:45:04.552540', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO68ZKgINynTjw8oLOmvyjkPX60IzTUiHP2uuX/5R/nZ4ZZgXeZgIUJar7hsXwekypDDbIa18jgYIHRQkwbfm/U='} 2023-06-22T16:49:53.819 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-22T16:49:53.830 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-06-22T16:49:53.830 INFO:teuthology.task.internal:roles: ubuntu@smithi202.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-06-22T16:49:53.831 INFO:teuthology.run_tasks:Running task console_log... 2023-06-22T16:49:53.927 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa38f0f0040>, signals=[15]) 2023-06-22T16:49:53.928 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-22T16:49:53.939 INFO:teuthology.task.internal:Opening connections... 2023-06-22T16:49:53.940 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T16:49:53.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T16:49:54.009 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T16:49:54.009 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T16:49:54.076 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-22T16:49:54.085 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2023-06-22T16:49:54.103 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2023-06-22T16:49:54.103 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2023-06-22T16:49:54.159 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="Red Hat Enterprise Linux" 2023-06-22T16:49:54.160 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="8.4 (Ootpa)" 2023-06-22T16:49:54.160 INFO:teuthology.orchestra.run.smithi047.stdout:ID="rhel" 2023-06-22T16:49:54.160 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="fedora" 2023-06-22T16:49:54.160 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="8.4" 2023-06-22T16:49:54.160 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el8" 2023-06-22T16:49:54.160 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-22T16:49:54.161 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2023-06-22T16:49:54.161 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-22T16:49:54.161 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://www.redhat.com/" 2023-06-22T16:49:54.161 INFO:teuthology.orchestra.run.smithi047.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-22T16:49:54.161 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-22T16:49:54.161 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T16:49:54.161 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-22T16:49:54.162 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-22T16:49:54.162 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-22T16:49:54.162 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-22T16:49:54.162 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2023-06-22T16:49:54.199 DEBUG:teuthology.orchestra.run.smithi202:> uname -m 2023-06-22T16:49:54.213 INFO:teuthology.orchestra.run.smithi202.stdout:x86_64 2023-06-22T16:49:54.213 DEBUG:teuthology.orchestra.run.smithi202:> cat /etc/os-release 2023-06-22T16:49:54.269 INFO:teuthology.orchestra.run.smithi202.stdout:NAME="Red Hat Enterprise Linux" 2023-06-22T16:49:54.269 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION="8.4 (Ootpa)" 2023-06-22T16:49:54.269 INFO:teuthology.orchestra.run.smithi202.stdout:ID="rhel" 2023-06-22T16:49:54.270 INFO:teuthology.orchestra.run.smithi202.stdout:ID_LIKE="fedora" 2023-06-22T16:49:54.270 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION_ID="8.4" 2023-06-22T16:49:54.270 INFO:teuthology.orchestra.run.smithi202.stdout:PLATFORM_ID="platform:el8" 2023-06-22T16:49:54.270 INFO:teuthology.orchestra.run.smithi202.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-22T16:49:54.270 INFO:teuthology.orchestra.run.smithi202.stdout:ANSI_COLOR="0;31" 2023-06-22T16:49:54.270 INFO:teuthology.orchestra.run.smithi202.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-22T16:49:54.270 INFO:teuthology.orchestra.run.smithi202.stdout:HOME_URL="https://www.redhat.com/" 2023-06-22T16:49:54.271 INFO:teuthology.orchestra.run.smithi202.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-22T16:49:54.271 INFO:teuthology.orchestra.run.smithi202.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-22T16:49:54.271 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T16:49:54.271 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-22T16:49:54.271 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-22T16:49:54.271 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-22T16:49:54.271 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-22T16:49:54.272 INFO:teuthology.lock.ops:Updating smithi202.front.sepia.ceph.com on lock server 2023-06-22T16:49:54.298 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-22T16:49:54.310 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-22T16:49:54.334 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-22T16:49:54.334 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2023-06-22T16:49:54.336 DEBUG:teuthology.orchestra.run.smithi202:> test '!' -e /home/ubuntu/cephtest 2023-06-22T16:49:54.350 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-22T16:49:54.367 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-22T16:49:54.367 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2023-06-22T16:49:54.392 DEBUG:teuthology.orchestra.run.smithi202:> test -z $(ls -A /var/lib/ceph) 2023-06-22T16:49:54.407 INFO:teuthology.orchestra.run.smithi047.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-22T16:49:54.408 INFO:teuthology.orchestra.run.smithi202.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-22T16:49:54.408 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-22T16:49:54.475 INFO:teuthology.run_tasks:Running task kernel... 2023-06-22T16:49:54.495 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-06-22T16:49:54.495 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-06-22T16:49:54.496 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-06-22T16:49:54.496 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-22T16:49:54.497 DEBUG:teuthology.orchestra.run.smithi202:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-22T16:49:54.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T16:49:54.510 DEBUG:teuthology.orchestra.run.smithi202:> uname -r 2023-06-22T16:49:54.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T16:49:54.513 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2023-06-22T16:49:54.567 INFO:teuthology.orchestra.run.smithi202.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-06-22T16:49:54.567 INFO:teuthology.task.kernel:Running kernel on smithi202: 4.18.0-305.25.1.el8_4.x86_64 2023-06-22T16:49:54.567 DEBUG:teuthology.orchestra.run.smithi202:> sudo yum install -y kernel 2023-06-22T16:49:54.570 INFO:teuthology.orchestra.run.smithi047.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-06-22T16:49:54.570 INFO:teuthology.task.kernel:Running kernel on smithi047: 4.18.0-305.25.1.el8_4.x86_64 2023-06-22T16:49:54.570 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2023-06-22T16:49:55.941 INFO:teuthology.orchestra.run.smithi202.stdout:Updating Subscription Management repositories. 2023-06-22T16:49:55.942 INFO:teuthology.orchestra.run.smithi202.stdout:Unable to read consumer identity 2023-06-22T16:49:56.075 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2023-06-22T16:49:56.075 INFO:teuthology.orchestra.run.smithi047.stdout:Unable to read consumer identity 2023-06-22T16:49:59.606 INFO:teuthology.orchestra.run.smithi047.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-06-22T16:49:59.699 INFO:teuthology.orchestra.run.smithi202.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-22T16:50:00.104 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 34 MB/s | 16 MB 00:00 2023-06-22T16:50:00.628 INFO:teuthology.orchestra.run.smithi202.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-06-22T16:50:03.592 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 1.7 MB/s | 24 kB 00:00 2023-06-22T16:50:03.890 INFO:teuthology.orchestra.run.smithi202.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-06-22T16:50:04.327 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-06-22T16:50:04.328 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-06-22T16:50:04.328 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-06-22T16:50:04.475 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2023-06-22T16:50:04.477 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2023-06-22T16:50:04.477 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2023-06-22T16:50:04.586 DEBUG:teuthology.orchestra.run.smithi047:> echo no | sudo yum reinstall kernel || true 2023-06-22T16:50:04.653 INFO:teuthology.orchestra.run.smithi202.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-06-22T16:50:04.653 INFO:teuthology.orchestra.run.smithi202.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-06-22T16:50:04.653 INFO:teuthology.orchestra.run.smithi202.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-06-22T16:50:04.810 INFO:teuthology.orchestra.run.smithi202.stdout:Dependencies resolved. 2023-06-22T16:50:04.811 INFO:teuthology.orchestra.run.smithi202.stdout:Nothing to do. 2023-06-22T16:50:04.811 INFO:teuthology.orchestra.run.smithi202.stdout:Complete! 2023-06-22T16:50:04.914 DEBUG:teuthology.orchestra.run.smithi202:> echo no | sudo yum reinstall kernel || true 2023-06-22T16:50:04.953 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2023-06-22T16:50:04.953 INFO:teuthology.orchestra.run.smithi047.stdout:Unable to read consumer identity 2023-06-22T16:50:05.258 INFO:teuthology.orchestra.run.smithi202.stdout:Updating Subscription Management repositories. 2023-06-22T16:50:05.258 INFO:teuthology.orchestra.run.smithi202.stdout:Unable to read consumer identity 2023-06-22T16:50:05.264 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:02 ago on Thu 22 Jun 2023 04:50:03 PM UTC. 2023-06-22T16:50:05.401 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-22T16:50:05.401 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-22T16:50:05.401 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-22T16:50:05.419 INFO:teuthology.orchestra.run.smithi047.stderr:Error: No packages marked for reinstall. 2023-06-22T16:50:05.469 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum reinstall -y kernel || true 2023-06-22T16:50:05.549 INFO:teuthology.orchestra.run.smithi202.stdout:Last metadata expiration check: 0:00:02 ago on Thu 22 Jun 2023 04:50:03 PM UTC. 2023-06-22T16:50:05.677 INFO:teuthology.orchestra.run.smithi202.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-22T16:50:05.677 INFO:teuthology.orchestra.run.smithi202.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-22T16:50:05.677 INFO:teuthology.orchestra.run.smithi202.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-22T16:50:05.709 INFO:teuthology.orchestra.run.smithi202.stderr:Error: No packages marked for reinstall. 2023-06-22T16:50:05.760 DEBUG:teuthology.orchestra.run.smithi202:> sudo yum reinstall -y kernel || true 2023-06-22T16:50:05.836 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2023-06-22T16:50:05.837 INFO:teuthology.orchestra.run.smithi047.stdout:Unable to read consumer identity 2023-06-22T16:50:06.103 INFO:teuthology.orchestra.run.smithi202.stdout:Updating Subscription Management repositories. 2023-06-22T16:50:06.104 INFO:teuthology.orchestra.run.smithi202.stdout:Unable to read consumer identity 2023-06-22T16:50:06.132 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:03 ago on Thu 22 Jun 2023 04:50:03 PM UTC. 2023-06-22T16:50:06.267 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-22T16:50:06.267 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-22T16:50:06.267 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-22T16:50:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:Error: No packages marked for reinstall. 2023-06-22T16:50:06.332 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2023-06-22T16:50:06.388 INFO:teuthology.orchestra.run.smithi202.stdout:Last metadata expiration check: 0:00:03 ago on Thu 22 Jun 2023 04:50:03 PM UTC. 2023-06-22T16:50:06.461 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.462 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.462 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.463 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-22T16:50:06.463 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-22T16:50:06.463 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-06-22T16:50:06.463 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2023-06-22T16:50:06.463 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2023-06-22T16:50:06.526 INFO:teuthology.orchestra.run.smithi202.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-22T16:50:06.526 INFO:teuthology.orchestra.run.smithi202.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-22T16:50:06.526 INFO:teuthology.orchestra.run.smithi202.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-22T16:50:06.539 INFO:teuthology.orchestra.run.smithi202.stderr:Error: No packages marked for reinstall. 2023-06-22T16:50:06.589 DEBUG:teuthology.orchestra.run.smithi202:> rpm -q kernel | sort -rV | head -n 1 2023-06-22T16:50:06.593 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.593 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.594 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2023-06-22T16:50:06.621 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2023-06-22T16:50:06.621 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2023-06-22T16:50:06.622 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.02 2023-06-22T16:50:06.622 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 99.el8_4.1 2023-06-22T16:50:06.622 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2023-06-22T16:50:06.622 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-06-22T16:50:06.622 INFO:teuthology.orchestra.run.smithi047.stdout:Group : System Environment/Base 2023-06-22T16:50:06.622 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 9264631 2023-06-22T16:50:06.622 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2023-06-22T16:50:06.623 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-06-22T16:50:06.623 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-06-22T16:50:06.623 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-06-22T16:50:06.623 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-06-22T16:50:06.623 INFO:teuthology.orchestra.run.smithi047.stdout:Relocations : (not relocatable) 2023-06-22T16:50:06.623 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : Red Hat, Inc. 2023-06-22T16:50:06.623 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : Red Hat, Inc. 2023-06-22T16:50:06.624 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-22T16:50:06.624 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2023-06-22T16:50:06.624 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2023-06-22T16:50:06.624 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T16:50:06.624 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-22T16:50:06.624 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-22T16:50:06.624 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-22T16:50:06.624 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2023-06-22T16:50:06.625 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T16:50:06.625 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2023-06-22T16:50:06.625 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-22T16:50:06.626 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-22T16:50:06.713 INFO:teuthology.orchestra.run.smithi202.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.714 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.714 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.714 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-22T16:50:06.715 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-22T16:50:06.715 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-06-22T16:50:06.715 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi202.front.sepia.ceph.com, path=None, version=distro) 2023-06-22T16:50:06.715 DEBUG:teuthology.orchestra.run.smithi202:> rpm -q kernel | sort -rV | head -n 1 2023-06-22T16:50:06.839 INFO:teuthology.orchestra.run.smithi202.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.840 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.840 DEBUG:teuthology.orchestra.run.smithi202:> sudo rpm -qi grub2-tools 2023-06-22T16:50:06.867 INFO:teuthology.orchestra.run.smithi202.stdout:Name : grub2-tools 2023-06-22T16:50:06.867 INFO:teuthology.orchestra.run.smithi202.stdout:Epoch : 1 2023-06-22T16:50:06.867 INFO:teuthology.orchestra.run.smithi202.stdout:Version : 2.02 2023-06-22T16:50:06.867 INFO:teuthology.orchestra.run.smithi202.stdout:Release : 99.el8_4.1 2023-06-22T16:50:06.867 INFO:teuthology.orchestra.run.smithi202.stdout:Architecture: x86_64 2023-06-22T16:50:06.868 INFO:teuthology.orchestra.run.smithi202.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-06-22T16:50:06.868 INFO:teuthology.orchestra.run.smithi202.stdout:Group : System Environment/Base 2023-06-22T16:50:06.868 INFO:teuthology.orchestra.run.smithi202.stdout:Size : 9264631 2023-06-22T16:50:06.868 INFO:teuthology.orchestra.run.smithi202.stdout:License : GPLv3+ 2023-06-22T16:50:06.868 INFO:teuthology.orchestra.run.smithi202.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-06-22T16:50:06.868 INFO:teuthology.orchestra.run.smithi202.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-06-22T16:50:06.868 INFO:teuthology.orchestra.run.smithi202.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-06-22T16:50:06.869 INFO:teuthology.orchestra.run.smithi202.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-06-22T16:50:06.869 INFO:teuthology.orchestra.run.smithi202.stdout:Relocations : (not relocatable) 2023-06-22T16:50:06.869 INFO:teuthology.orchestra.run.smithi202.stdout:Packager : Red Hat, Inc. 2023-06-22T16:50:06.869 INFO:teuthology.orchestra.run.smithi202.stdout:Vendor : Red Hat, Inc. 2023-06-22T16:50:06.869 INFO:teuthology.orchestra.run.smithi202.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-22T16:50:06.869 INFO:teuthology.orchestra.run.smithi202.stdout:Summary : Support tools for GRUB. 2023-06-22T16:50:06.869 INFO:teuthology.orchestra.run.smithi202.stdout:Description : 2023-06-22T16:50:06.869 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T16:50:06.870 INFO:teuthology.orchestra.run.smithi202.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-22T16:50:06.870 INFO:teuthology.orchestra.run.smithi202.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-22T16:50:06.870 INFO:teuthology.orchestra.run.smithi202.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-22T16:50:06.870 INFO:teuthology.orchestra.run.smithi202.stdout:hardware devices. 2023-06-22T16:50:06.870 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T16:50:06.870 INFO:teuthology.orchestra.run.smithi202.stdout:This subpackage provides tools for support of all platforms. 2023-06-22T16:50:06.871 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-22T16:50:06.871 INFO:teuthology.task.kernel:Updating grub on smithi202 to boot 4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:06.871 DEBUG:teuthology.orchestra.run.smithi202:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-22T16:50:07.707 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2023-06-22T16:50:08.308 INFO:teuthology.orchestra.run.smithi202.stderr:Generating grub configuration file ... 2023-06-22T16:50:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:done 2023-06-22T16:50:09.901 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2023-06-22T16:50:09.923 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.KZIe5oJ6Q7 2023-06-22T16:50:09.923 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KZIe5oJ6Q7 2023-06-22T16:50:10.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.KZIe5oJ6Q7 2023-06-22T16:50:10.136 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.KZIe5oJ6Q7 is 6KB 2023-06-22T16:50:10.154 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.KZIe5oJ6Q7 2023-06-22T16:50:10.167 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2023-06-22T16:50:10.234 INFO:teuthology.orchestra.run.smithi047.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-06-22T16:50:10.234 INFO:teuthology.orchestra.run.smithi047.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-06-22T16:50:10.234 INFO:teuthology.orchestra.run.smithi047.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-06-22T16:50:10.234 INFO:teuthology.orchestra.run.smithi047.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-06-22T16:50:10.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:10.391 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2023-06-22T16:50:10.534 INFO:teuthology.orchestra.run.smithi202.stderr:done 2023-06-22T16:50:10.536 DEBUG:teuthology.orchestra.run.smithi202:> mktemp 2023-06-22T16:50:10.567 INFO:teuthology.orchestra.run.smithi202.stdout:/tmp/tmp.sV0yt29tLF 2023-06-22T16:50:10.568 DEBUG:teuthology.orchestra.run.smithi202:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.sV0yt29tLF 2023-06-22T16:50:10.637 DEBUG:teuthology.orchestra.run.smithi202:> sudo chmod 0666 /tmp/tmp.sV0yt29tLF 2023-06-22T16:50:10.801 DEBUG:teuthology.orchestra.remote:smithi202:/tmp/tmp.sV0yt29tLF is 6KB 2023-06-22T16:50:10.848 DEBUG:teuthology.orchestra.run.smithi202:> rm -fr /tmp/tmp.sV0yt29tLF 2023-06-22T16:50:10.861 DEBUG:teuthology.orchestra.run.smithi202:> sudo /bin/ls /boot/loader/entries || true 2023-06-22T16:50:10.932 INFO:teuthology.orchestra.run.smithi202.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-06-22T16:50:10.932 INFO:teuthology.orchestra.run.smithi202.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-06-22T16:50:10.932 INFO:teuthology.orchestra.run.smithi202.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-06-22T16:50:10.932 INFO:teuthology.orchestra.run.smithi202.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-06-22T16:50:10.933 DEBUG:teuthology.orchestra.run.smithi202:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-06-22T16:50:11.083 DEBUG:teuthology.orchestra.run.smithi202:> sudo shutdown -r now 2023-06-22T16:50:40.420 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-06-22T16:50:40.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2023-06-22T16:50:40.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T16:50:41.087 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-06-22T16:50:41.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi202.front.sepia.ceph.com' 2023-06-22T16:50:41.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T16:50:58.800 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2023-06-22T16:50:59.568 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-06-22T16:51:28.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2023-06-22T16:51:28.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T16:51:29.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi202.front.sepia.ceph.com' 2023-06-22T16:51:29.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T16:51:44.989 DEBUG:teuthology.orchestra.run.smithi047:> true 2023-06-22T16:51:45.262 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2023-06-22T16:51:45.263 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-06-22T16:51:45.263 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2023-06-22T16:51:45.284 INFO:teuthology.orchestra.run.smithi047.stdout:4.18.0-425.3.1.el8.x86_64 2023-06-22T16:51:45.284 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-06-22T16:51:45.284 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-22T16:51:45.284 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-06-22T16:51:46.285 DEBUG:teuthology.parallel:result is None 2023-06-22T16:51:47.952 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-06-22T16:52:17.953 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi202.front.sepia.ceph.com' 2023-06-22T16:52:17.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T16:52:18.314 DEBUG:teuthology.orchestra.run.smithi202:> true 2023-06-22T16:52:18.600 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi202.front.sepia.ceph.com' 2023-06-22T16:52:18.601 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-06-22T16:52:18.601 DEBUG:teuthology.orchestra.run.smithi202:> uname -r 2023-06-22T16:52:18.615 INFO:teuthology.orchestra.run.smithi202.stdout:4.18.0-425.3.1.el8.x86_64 2023-06-22T16:52:18.615 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-06-22T16:52:18.615 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-22T16:52:18.616 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-06-22T16:52:19.616 DEBUG:teuthology.parallel:result is None 2023-06-22T16:52:19.617 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-22T16:52:19.635 INFO:teuthology.task.internal:Creating test directory... 2023-06-22T16:52:19.635 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-22T16:52:19.640 DEBUG:teuthology.orchestra.run.smithi202:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-22T16:52:19.656 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-22T16:52:19.666 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-22T16:52:19.709 INFO:teuthology.task.internal:Creating archive directory... 2023-06-22T16:52:19.709 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-22T16:52:19.712 DEBUG:teuthology.orchestra.run.smithi202:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-22T16:52:19.739 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-22T16:52:19.771 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-22T16:52:19.771 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-22T16:52:19.775 DEBUG:teuthology.orchestra.run.smithi202:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-22T16:52:19.846 INFO:teuthology.orchestra.run.smithi202.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-22T16:52:19.864 INFO:teuthology.orchestra.run.smithi202.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-22T16:52:19.898 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-22T16:52:19.916 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-22T16:52:19.917 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-22T16:52:19.924 INFO:teuthology.task.internal:Configuring sudo... 2023-06-22T16:52:19.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-22T16:52:19.927 DEBUG:teuthology.orchestra.run.smithi202:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-22T16:52:19.952 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-22T16:52:19.991 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-22T16:52:19.992 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-22T16:52:19.994 DEBUG:teuthology.orchestra.run.smithi202:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-22T16:52:20.009 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-22T16:52:20.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-22T16:52:20.133 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-22T16:52:20.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-22T16:52:20.257 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T16:52:20.258 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-22T16:52:20.322 DEBUG:teuthology.orchestra.run.smithi202:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-22T16:52:20.369 DEBUG:teuthology.orchestra.run.smithi202:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-22T16:52:20.433 DEBUG:teuthology.orchestra.run.smithi202:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-22T16:52:20.473 DEBUG:teuthology.orchestra.run.smithi202:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-22T16:52:20.514 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T16:52:20.514 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-22T16:52:20.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2023-06-22T16:52:20.580 DEBUG:teuthology.orchestra.run.smithi202:> sudo service rsyslog restart 2023-06-22T16:52:20.634 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-22T16:52:20.673 INFO:teuthology.orchestra.run.smithi202.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-22T16:52:21.031 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-22T16:52:21.039 INFO:teuthology.task.internal:Starting timer... 2023-06-22T16:52:21.039 INFO:teuthology.run_tasks:Running task pcp... 2023-06-22T16:52:21.097 INFO:teuthology.run_tasks:Running task selinux... 2023-06-22T16:52:21.149 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-06-22T16:52:21.150 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2023-06-22T16:52:21.218 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: [ OK ] 2023-06-22T16:52:21.220 DEBUG:teuthology.orchestra.run.smithi202:> sudo service auditd rotate 2023-06-22T16:52:21.280 INFO:teuthology.orchestra.run.smithi202.stdout:Rotating logs: [ OK ] 2023-06-22T16:52:21.281 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-22T16:52:21.281 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2023-06-22T16:52:21.305 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2023-06-22T16:52:21.306 DEBUG:teuthology.orchestra.run.smithi202:> /usr/sbin/getenforce 2023-06-22T16:52:21.328 INFO:teuthology.orchestra.run.smithi202.stdout:Permissive 2023-06-22T16:52:21.329 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi202.front.sepia.ceph.com': 'permissive'} 2023-06-22T16:52:21.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-22T16:52:21.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T16:52:21.372 DEBUG:teuthology.orchestra.run.smithi202:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-22T16:52:21.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T16:52:21.398 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-22T16:52:21.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2023-06-22T16:52:21.437 DEBUG:teuthology.orchestra.run.smithi202:> sudo /usr/sbin/setenforce permissive 2023-06-22T16:52:21.463 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-22T16:52:21.473 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-06-22T16:52:21.473 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-22T16:52:21.492 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-22T16:52:21.494 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi202.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-22T17:00:12.931 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi202.front.sepia.ceph.com')] 2023-06-22T17:00:12.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2023-06-22T17:00:12.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:00:13.002 DEBUG:teuthology.orchestra.run.smithi047:> true 2023-06-22T17:00:13.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi202.front.sepia.ceph.com' 2023-06-22T17:00:13.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:00:13.140 DEBUG:teuthology.orchestra.run.smithi202:> true 2023-06-22T17:00:13.203 INFO:teuthology.run_tasks:Running task clock... 2023-06-22T17:00:13.218 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-22T17:00:13.219 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-22T17:00:13.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-22T17:00:13.221 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-22T17:00:13.221 DEBUG:teuthology.orchestra.run.smithi202:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-22T17:00:13.250 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-22T17:00:13.265 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-22T17:00:13.272 INFO:teuthology.orchestra.run.smithi202.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-22T17:00:13.287 INFO:teuthology.orchestra.run.smithi202.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-22T17:00:13.303 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2023-06-22T17:00:13.314 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2023-06-22T17:00:13.329 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-22T17:00:13.329 INFO:teuthology.orchestra.run.smithi202.stderr:sudo: ntpd: command not found 2023-06-22T17:00:13.340 INFO:teuthology.orchestra.run.smithi202.stdout:506 Cannot talk to daemon 2023-06-22T17:00:13.343 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-22T17:00:13.353 INFO:teuthology.orchestra.run.smithi202.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-22T17:00:13.367 INFO:teuthology.orchestra.run.smithi202.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-22T17:00:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:bash: ntpq: command not found 2023-06-22T17:00:13.412 INFO:teuthology.orchestra.run.smithi047.stdout:210 Number of sources = 4 2023-06-22T17:00:13.412 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-22T17:00:13.412 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2023-06-22T17:00:13.412 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:00:13.412 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:00:13.413 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:00:13.413 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:00:13.429 INFO:teuthology.orchestra.run.smithi202.stderr:bash: ntpq: command not found 2023-06-22T17:00:13.431 INFO:teuthology.orchestra.run.smithi202.stdout:210 Number of sources = 4 2023-06-22T17:00:13.432 INFO:teuthology.orchestra.run.smithi202.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-22T17:00:13.432 INFO:teuthology.orchestra.run.smithi202.stdout:=============================================================================== 2023-06-22T17:00:13.432 INFO:teuthology.orchestra.run.smithi202.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:00:13.432 INFO:teuthology.orchestra.run.smithi202.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:00:13.432 INFO:teuthology.orchestra.run.smithi202.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:00:13.432 INFO:teuthology.orchestra.run.smithi202.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:00:13.433 INFO:teuthology.run_tasks:Running task install... 2023-06-22T17:00:13.442 DEBUG:teuthology.task.install:project ceph 2023-06-22T17:00:13.443 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}} 2023-06-22T17:00:13.444 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'} 2023-06-22T17:00:13.444 INFO:teuthology.task.install:Using flavor: default 2023-06-22T17:00:13.455 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-22T17:00:13.456 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-22T17:00:13.457 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-22T17:00:13.458 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T17:00:13.461 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-22T17:00:13.462 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T17:00:13.604 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-22T17:00:13.604 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-22T17:00:13.624 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-22T17:00:13.625 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-22T17:00:13.711 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-22T17:00:13.712 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:00:13.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-22T17:00:13.739 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-22T17:00:13.739 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-22T17:00:13.787 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-22T17:00:13.787 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:00:13.787 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-22T17:00:13.805 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-22T17:00:13.812 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-22T17:00:13.812 DEBUG:teuthology.orchestra.run.smithi202:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-22T17:00:13.876 DEBUG:teuthology.orchestra.run.smithi202:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-22T17:00:13.881 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-22T17:00:13.945 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2023-06-22T17:00:13.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2023-06-22T17:00:13.949 DEBUG:teuthology.orchestra.run.smithi202:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-22T17:00:14.013 INFO:teuthology.orchestra.run.smithi202.stdout:check_obsoletes = 1 2023-06-22T17:00:14.014 DEBUG:teuthology.orchestra.run.smithi202:> sudo yum clean all 2023-06-22T17:00:14.358 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2023-06-22T17:00:14.403 INFO:teuthology.orchestra.run.smithi202.stdout:Updating Subscription Management repositories. 2023-06-22T17:00:14.880 INFO:teuthology.orchestra.run.smithi047.stdout:44 files removed 2023-06-22T17:00:14.919 INFO:teuthology.orchestra.run.smithi202.stdout:44 files removed 2023-06-22T17:00:14.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-22T17:00:14.975 DEBUG:teuthology.orchestra.run.smithi202:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-22T17:00:15.302 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2023-06-22T17:00:15.324 INFO:teuthology.orchestra.run.smithi202.stdout:Updating Subscription Management repositories. 2023-06-22T17:00:16.208 INFO:teuthology.orchestra.run.smithi202.stdout:ceph packages for x86_64 203 kB/s | 78 kB 00:00 2023-06-22T17:00:16.385 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 136 kB/s | 78 kB 00:00 2023-06-22T17:00:16.720 INFO:teuthology.orchestra.run.smithi202.stdout:ceph noarch packages 19 kB/s | 9.2 kB 00:00 2023-06-22T17:00:16.756 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 28 kB/s | 9.2 kB 00:00 2023-06-22T17:00:17.081 INFO:teuthology.orchestra.run.smithi202.stdout:ceph source packages 4.4 kB/s | 1.4 kB 00:00 2023-06-22T17:00:17.249 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 3.2 kB/s | 1.4 kB 00:00 2023-06-22T17:00:19.557 INFO:teuthology.orchestra.run.smithi202.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-22T17:00:19.725 INFO:teuthology.orchestra.run.smithi047.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-22T17:00:20.654 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-06-22T17:00:24.196 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 1.7 MB/s | 24 kB 00:00 2023-06-22T17:00:24.650 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 104 MB/s | 38 MB 00:00 2023-06-22T17:00:30.657 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 102 MB/s | 33 MB 00:00 2023-06-22T17:00:39.019 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-22T17:00:39.020 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-22T17:00:39.031 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-22T17:00:39.032 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-22T17:00:39.033 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-22T17:00:39.034 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-22T17:00:39.036 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-22T17:00:39.038 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-22T17:00:39.039 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-22T17:00:39.546 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2023-06-22T17:00:39.558 INFO:teuthology.orchestra.run.smithi047.stdout:====================================================================================================================== 2023-06-22T17:00:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2023-06-22T17:00:39.559 INFO:teuthology.orchestra.run.smithi047.stdout:====================================================================================================================== 2023-06-22T17:00:39.559 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2023-06-22T17:00:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-22T17:00:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-22T17:00:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-22T17:00:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-22T17:00:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-22T17:00:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-22T17:00:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-22T17:00:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-22T17:00:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-22T17:00:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-22T17:00:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-22T17:00:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-22T17:00:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-22T17:00:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-22T17:00:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-22T17:00:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-22T17:00:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-22T17:00:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-22T17:00:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-22T17:00:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-22T17:00:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-22T17:00:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-22T17:00:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-22T17:00:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-22T17:00:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-22T17:00:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-22T17:00:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-22T17:00:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-22T17:00:39.564 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-22T17:00:39.564 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-22T17:00:39.564 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-22T17:00:39.564 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-22T17:00:39.564 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-22T17:00:39.564 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2023-06-22T17:00:39.565 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-22T17:00:39.565 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-22T17:00:39.565 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2023-06-22T17:00:39.565 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-22T17:00:39.565 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-22T17:00:39.566 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-22T17:00:39.566 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-22T17:00:39.566 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-22T17:00:39.567 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-22T17:00:39.567 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-22T17:00:39.567 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-22T17:00:39.568 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-22T17:00:39.568 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-22T17:00:39.568 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-22T17:00:39.568 INFO:teuthology.orchestra.run.smithi047.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-22T17:00:39.569 INFO:teuthology.orchestra.run.smithi047.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-22T17:00:39.569 INFO:teuthology.orchestra.run.smithi047.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-22T17:00:39.569 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-22T17:00:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-22T17:00:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-22T17:00:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-22T17:00:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-22T17:00:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-22T17:00:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-22T17:00:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-22T17:00:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-22T17:00:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-22T17:00:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-22T17:00:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-22T17:00:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-22T17:00:39.571 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-22T17:00:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-22T17:00:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-22T17:00:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-22T17:00:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-22T17:00:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-22T17:00:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-22T17:00:39.572 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-22T17:00:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-22T17:00:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-22T17:00:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-22T17:00:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-22T17:00:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-22T17:00:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-22T17:00:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-22T17:00:39.573 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-22T17:00:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-22T17:00:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-22T17:00:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-22T17:00:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-22T17:00:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-22T17:00:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-22T17:00:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-22T17:00:39.574 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-22T17:00:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-22T17:00:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-22T17:00:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-22T17:00:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-22T17:00:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-22T17:00:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-22T17:00:39.575 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-22T17:00:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-22T17:00:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-22T17:00:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-22T17:00:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-22T17:00:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-22T17:00:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-22T17:00:39.576 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-22T17:00:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-22T17:00:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-22T17:00:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-22T17:00:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-22T17:00:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-22T17:00:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-22T17:00:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-22T17:00:39.577 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-22T17:00:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-22T17:00:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-22T17:00:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-22T17:00:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-22T17:00:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-22T17:00:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-22T17:00:39.578 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-22T17:00:39.579 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-22T17:00:39.579 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-22T17:00:39.579 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-22T17:00:39.579 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-22T17:00:39.579 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-22T17:00:39.579 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:00:39.579 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2023-06-22T17:00:39.579 INFO:teuthology.orchestra.run.smithi047.stdout:====================================================================================================================== 2023-06-22T17:00:39.580 INFO:teuthology.orchestra.run.smithi047.stdout:Install 118 Packages 2023-06-22T17:00:39.580 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 2 Packages 2023-06-22T17:00:39.580 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:00:39.580 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 201 M 2023-06-22T17:00:39.581 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2023-06-22T17:00:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 27 kB/s | 6.4 kB 00:00 2023-06-22T17:00:40.479 INFO:teuthology.orchestra.run.smithi047.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 2.1 MB/s | 873 kB 00:00 2023-06-22T17:00:40.541 INFO:teuthology.orchestra.run.smithi047.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 2.5 MB/s | 162 kB 00:00 2023-06-22T17:00:40.894 INFO:teuthology.orchestra.run.smithi047.stdout:(4/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 4.9 MB/s | 5.2 MB 00:01 2023-06-22T17:00:40.902 INFO:teuthology.orchestra.run.smithi047.stdout:(5/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 6.4 MB/s | 2.3 MB 00:00 2023-06-22T17:00:41.636 INFO:teuthology.orchestra.run.smithi047.stdout:(6/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 2.1 MB/s | 1.6 MB 00:00 2023-06-22T17:00:42.777 INFO:teuthology.orchestra.run.smithi047.stdout:(7/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 2.1 MB/s | 3.9 MB 00:01 2023-06-22T17:00:45.644 INFO:teuthology.orchestra.run.smithi047.stdout:(8/120): ceph-common-17.2.6-719.g42ddc46e.el8.x 4.0 MB/s | 23 MB 00:05 2023-06-22T17:00:45.696 INFO:teuthology.orchestra.run.smithi047.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 473 kB/s | 24 kB 00:00 2023-06-22T17:00:48.540 INFO:teuthology.orchestra.run.smithi047.stdout:(10/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86 2.4 MB/s | 17 MB 00:06 2023-06-22T17:00:48.759 INFO:teuthology.orchestra.run.smithi047.stdout:(11/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8 2.8 MB/s | 17 MB 00:05 2023-06-22T17:00:48.761 INFO:teuthology.orchestra.run.smithi047.stdout:(12/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 1.1 MB/s | 255 kB 00:00 2023-06-22T17:00:48.808 INFO:teuthology.orchestra.run.smithi047.stdout:(13/120): libcephfs-devel-17.2.6-719.g42ddc46e. 510 kB/s | 25 kB 00:00 2023-06-22T17:00:48.858 INFO:teuthology.orchestra.run.smithi047.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 3.4 MB/s | 174 kB 00:00 2023-06-22T17:00:48.907 INFO:teuthology.orchestra.run.smithi047.stdout:(15/120): librados-devel-17.2.6-719.g42ddc46e.e 2.6 MB/s | 130 kB 00:00 2023-06-22T17:00:49.005 INFO:teuthology.orchestra.run.smithi047.stdout:(16/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 3.2 MB/s | 794 kB 00:00 2023-06-22T17:00:49.058 INFO:teuthology.orchestra.run.smithi047.stdout:(17/120): libradosstriper1-17.2.6-719.g42ddc46e 3.3 MB/s | 506 kB 00:00 2023-06-22T17:00:49.111 INFO:teuthology.orchestra.run.smithi047.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 846 kB/s | 45 kB 00:00 2023-06-22T17:00:49.164 INFO:teuthology.orchestra.run.smithi047.stdout:(19/120): python3-ceph-common-17.2.6-719.g42ddc 2.1 MB/s | 115 kB 00:00 2023-06-22T17:00:49.217 INFO:teuthology.orchestra.run.smithi047.stdout:(20/120): python3-cephfs-17.2.6-719.g42ddc46e.e 3.5 MB/s | 190 kB 00:00 2023-06-22T17:00:49.320 INFO:teuthology.orchestra.run.smithi047.stdout:(21/120): python3-rados-17.2.6-719.g42ddc46e.el 3.5 MB/s | 371 kB 00:00 2023-06-22T17:00:49.427 INFO:teuthology.orchestra.run.smithi047.stdout:(22/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 3.3 MB/s | 357 kB 00:00 2023-06-22T17:00:49.479 INFO:teuthology.orchestra.run.smithi047.stdout:(23/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 2.0 MB/s | 107 kB 00:00 2023-06-22T17:00:49.533 INFO:teuthology.orchestra.run.smithi047.stdout:(24/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 1.6 MB/s | 91 kB 00:00 2023-06-22T17:00:50.446 INFO:teuthology.orchestra.run.smithi047.stdout:(25/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 3.8 MB/s | 3.5 MB 00:00 2023-06-22T17:00:50.500 INFO:teuthology.orchestra.run.smithi047.stdout:(26/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 3.7 MB/s | 5.5 MB 00:01 2023-06-22T17:00:50.502 INFO:teuthology.orchestra.run.smithi047.stdout:(27/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 3.1 MB/s | 173 kB 00:00 2023-06-22T17:00:50.553 INFO:teuthology.orchestra.run.smithi047.stdout:(28/120): ceph-grafana-dashboards-17.2.6-719.g4 434 kB/s | 23 kB 00:00 2023-06-22T17:00:50.556 INFO:teuthology.orchestra.run.smithi047.stdout:(29/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 2.2 MB/s | 121 kB 00:00 2023-06-22T17:00:51.021 INFO:teuthology.orchestra.run.smithi047.stdout:(30/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 3.6 MB/s | 1.7 MB 00:00 2023-06-22T17:00:51.130 INFO:teuthology.orchestra.run.smithi047.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 2.1 MB/s | 238 kB 00:00 2023-06-22T17:00:51.186 INFO:teuthology.orchestra.run.smithi047.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 844 kB/s | 47 kB 00:00 2023-06-22T17:00:51.239 INFO:teuthology.orchestra.run.smithi047.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 277 kB/s | 14 kB 00:00 2023-06-22T17:00:51.347 INFO:teuthology.orchestra.run.smithi047.stdout:(34/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 2.4 MB/s | 264 kB 00:00 2023-06-22T17:00:51.403 INFO:teuthology.orchestra.run.smithi047.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 1.4 MB/s | 80 kB 00:00 2023-06-22T17:00:51.457 INFO:teuthology.orchestra.run.smithi047.stdout:(36/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 331 kB/s | 18 kB 00:00 2023-06-22T17:00:51.785 INFO:teuthology.orchestra.run.smithi047.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 933 kB/s | 306 kB 00:00 2023-06-22T17:00:51.842 INFO:teuthology.orchestra.run.smithi047.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-06-22T17:00:51.898 INFO:teuthology.orchestra.run.smithi047.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-06-22T17:00:52.014 INFO:teuthology.orchestra.run.smithi047.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2023-06-22T17:00:52.065 INFO:teuthology.orchestra.run.smithi047.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 865 kB/s | 44 kB 00:00 2023-06-22T17:00:52.124 INFO:teuthology.orchestra.run.smithi047.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2023-06-22T17:00:52.173 INFO:teuthology.orchestra.run.smithi047.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 681 kB/s | 33 kB 00:00 2023-06-22T17:00:52.222 INFO:teuthology.orchestra.run.smithi047.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 332 kB/s | 16 kB 00:00 2023-06-22T17:00:52.281 INFO:teuthology.orchestra.run.smithi047.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-06-22T17:00:52.312 INFO:teuthology.orchestra.run.smithi047.stdout:(46/120): ceph-mgr-diskprediction-local-17.2.6- 4.2 MB/s | 7.4 MB 00:01 2023-06-22T17:00:52.352 INFO:teuthology.orchestra.run.smithi047.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 5.3 MB/s | 384 kB 00:00 2023-06-22T17:00:52.401 INFO:teuthology.orchestra.run.smithi047.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 229 kB/s | 11 kB 00:00 2023-06-22T17:00:52.452 INFO:teuthology.orchestra.run.smithi047.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 344 kB/s | 18 kB 00:00 2023-06-22T17:00:52.552 INFO:teuthology.orchestra.run.smithi047.stdout:(50/120): python3-google-auth-1.1.1-10.el8.noar 338 kB/s | 81 kB 00:00 2023-06-22T17:00:52.571 INFO:teuthology.orchestra.run.smithi047.stdout:(51/120): python3-kubernetes-11.0.0-6.el8.noarc 9.8 MB/s | 1.2 MB 00:00 2023-06-22T17:00:52.621 INFO:teuthology.orchestra.run.smithi047.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-06-22T17:00:52.629 INFO:teuthology.orchestra.run.smithi047.stdout:(53/120): python3-logutils-0.3.5-11.el8.noarch. 636 kB/s | 49 kB 00:00 2023-06-22T17:00:52.670 INFO:teuthology.orchestra.run.smithi047.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-06-22T17:00:52.720 INFO:teuthology.orchestra.run.smithi047.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 315 kB/s | 16 kB 00:00 2023-06-22T17:00:52.769 INFO:teuthology.orchestra.run.smithi047.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 686 kB/s | 34 kB 00:00 2023-06-22T17:00:52.792 INFO:teuthology.orchestra.run.smithi047.stdout:(57/120): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-06-22T17:00:52.822 INFO:teuthology.orchestra.run.smithi047.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 3.6 MB/s | 196 kB 00:00 2023-06-22T17:00:52.848 INFO:teuthology.orchestra.run.smithi047.stdout:(59/120): python3-rsa-4.8-1.el8.noarch.rpm 1.1 MB/s | 61 kB 00:00 2023-06-22T17:00:52.871 INFO:teuthology.orchestra.run.smithi047.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 384 kB/s | 19 kB 00:00 2023-06-22T17:00:52.899 INFO:teuthology.orchestra.run.smithi047.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 475 kB/s | 24 kB 00:00 2023-06-22T17:00:52.919 INFO:teuthology.orchestra.run.smithi047.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 606 kB/s | 29 kB 00:00 2023-06-22T17:00:52.949 INFO:teuthology.orchestra.run.smithi047.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 553 kB/s | 27 kB 00:00 2023-06-22T17:00:52.975 INFO:teuthology.orchestra.run.smithi047.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 4.5 MB/s | 255 kB 00:00 2023-06-22T17:00:53.026 INFO:teuthology.orchestra.run.smithi047.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 3.2 MB/s | 251 kB 00:00 2023-06-22T17:00:53.026 INFO:teuthology.orchestra.run.smithi047.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-06-22T17:00:53.076 INFO:teuthology.orchestra.run.smithi047.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-06-22T17:00:53.077 INFO:teuthology.orchestra.run.smithi047.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 455 kB/s | 23 kB 00:00 2023-06-22T17:00:53.082 INFO:teuthology.orchestra.run.smithi047.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-06-22T17:00:53.116 INFO:teuthology.orchestra.run.smithi047.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 545 kB/s | 19 kB 00:00 2023-06-22T17:00:53.131 INFO:teuthology.orchestra.run.smithi047.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.9 MB/s | 43 kB 00:00 2023-06-22T17:00:53.147 INFO:teuthology.orchestra.run.smithi047.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 9.8 MB/s | 155 kB 00:00 2023-06-22T17:00:53.163 INFO:teuthology.orchestra.run.smithi047.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 14 MB/s | 220 kB 00:00 2023-06-22T17:00:53.177 INFO:teuthology.orchestra.run.smithi047.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.3 MB/s | 48 kB 00:00 2023-06-22T17:00:53.196 INFO:teuthology.orchestra.run.smithi047.stdout:(75/120): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2023-06-22T17:00:53.200 INFO:teuthology.orchestra.run.smithi047.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 10 MB/s | 229 kB 00:00 2023-06-22T17:00:53.227 INFO:teuthology.orchestra.run.smithi047.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 1.4 MB/s | 43 kB 00:00 2023-06-22T17:00:53.239 INFO:teuthology.orchestra.run.smithi047.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.5 MB/s | 18 kB 00:00 2023-06-22T17:00:53.248 INFO:teuthology.orchestra.run.smithi047.stdout:(79/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 48 MB/s | 2.3 MB 00:00 2023-06-22T17:00:53.260 INFO:teuthology.orchestra.run.smithi047.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 29 MB/s | 605 kB 00:00 2023-06-22T17:00:53.262 INFO:teuthology.orchestra.run.smithi047.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 5.9 MB/s | 83 kB 00:00 2023-06-22T17:00:53.273 INFO:teuthology.orchestra.run.smithi047.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 4.3 MB/s | 59 kB 00:00 2023-06-22T17:00:53.277 INFO:teuthology.orchestra.run.smithi047.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 11 MB/s | 169 kB 00:00 2023-06-22T17:00:53.286 INFO:teuthology.orchestra.run.smithi047.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.9 MB/s | 38 kB 00:00 2023-06-22T17:00:53.292 INFO:teuthology.orchestra.run.smithi047.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.9 MB/s | 86 kB 00:00 2023-06-22T17:00:53.299 INFO:teuthology.orchestra.run.smithi047.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.5 MB/s | 58 kB 00:00 2023-06-22T17:00:53.304 INFO:teuthology.orchestra.run.smithi047.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.7 MB/s | 47 kB 00:00 2023-06-22T17:00:53.320 INFO:teuthology.orchestra.run.smithi047.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 26 MB/s | 559 kB 00:00 2023-06-22T17:00:53.329 INFO:teuthology.orchestra.run.smithi047.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 25 MB/s | 639 kB 00:00 2023-06-22T17:00:53.333 INFO:teuthology.orchestra.run.smithi047.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 5.4 MB/s | 70 kB 00:00 2023-06-22T17:00:53.345 INFO:teuthology.orchestra.run.smithi047.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 13 MB/s | 200 kB 00:00 2023-06-22T17:00:53.347 INFO:teuthology.orchestra.run.smithi047.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 7.0 MB/s | 97 kB 00:00 2023-06-22T17:00:53.362 INFO:teuthology.orchestra.run.smithi047.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 2.8 MB/s | 44 kB 00:00 2023-06-22T17:00:53.375 INFO:teuthology.orchestra.run.smithi047.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.1 MB/s | 27 kB 00:00 2023-06-22T17:00:53.390 INFO:teuthology.orchestra.run.smithi047.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 11 MB/s | 161 kB 00:00 2023-06-22T17:00:53.406 INFO:teuthology.orchestra.run.smithi047.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 19 MB/s | 320 kB 00:00 2023-06-22T17:00:53.792 INFO:teuthology.orchestra.run.smithi047.stdout:(97/120): ceph-test-17.2.6-719.g42ddc46e.el8.x8 6.4 MB/s | 52 MB 00:08 2023-06-22T17:00:53.811 INFO:teuthology.orchestra.run.smithi047.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 1.1 MB/s | 457 kB 00:00 2023-06-22T17:00:53.827 INFO:teuthology.orchestra.run.smithi047.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.4 MB/s | 39 kB 00:00 2023-06-22T17:00:53.856 INFO:teuthology.orchestra.run.smithi047.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 23 MB/s | 688 kB 00:00 2023-06-22T17:00:53.876 INFO:teuthology.orchestra.run.smithi047.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 13 MB/s | 259 kB 00:00 2023-06-22T17:00:53.895 INFO:teuthology.orchestra.run.smithi047.stdout:(102/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 12 MB/s | 225 kB 00:00 2023-06-22T17:00:53.920 INFO:teuthology.orchestra.run.smithi047.stdout:(103/120): python3-babel-2.5.1-5.el8.noarch.rpm 38 MB/s | 4.8 MB 00:00 2023-06-22T17:00:53.922 INFO:teuthology.orchestra.run.smithi047.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 4.6 MB/s | 126 kB 00:00 2023-06-22T17:00:53.939 INFO:teuthology.orchestra.run.smithi047.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 11 MB/s | 187 kB 00:00 2023-06-22T17:00:53.962 INFO:teuthology.orchestra.run.smithi047.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 23 MB/s | 538 kB 00:00 2023-06-22T17:00:54.086 INFO:teuthology.orchestra.run.smithi047.stdout:(107/120): python3-scipy-1.0.0-20.module+el8.1. 20 MB/s | 15 MB 00:00 2023-06-22T17:00:54.091 INFO:teuthology.orchestra.run.smithi047.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 2.3 MB/s | 302 kB 00:00 2023-06-22T17:00:54.112 INFO:teuthology.orchestra.run.smithi047.stdout:(109/120): python3-numpy-1.14.3-9.el8.x86_64.rp 19 MB/s | 3.7 MB 00:00 2023-06-22T17:00:54.113 INFO:teuthology.orchestra.run.smithi047.stdout:(110/120): python3-mako-1.0.6-13.el8.noarch.rpm 5.7 MB/s | 157 kB 00:00 2023-06-22T17:00:54.114 INFO:teuthology.orchestra.run.smithi047.stdout:(111/120): python3-pyasn1-modules-0.3.7-6.el8.n 4.7 MB/s | 110 kB 00:00 2023-06-22T17:00:54.130 INFO:teuthology.orchestra.run.smithi047.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.2 MB/s | 20 kB 00:00 2023-06-22T17:00:54.132 INFO:teuthology.orchestra.run.smithi047.stdout:(113/120): librdkafka-0.11.4-1.el8.x86_64.rpm 17 MB/s | 353 kB 00:00 2023-06-22T17:00:54.140 INFO:teuthology.orchestra.run.smithi047.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 27 MB/s | 709 kB 00:00 2023-06-22T17:00:54.146 INFO:teuthology.orchestra.run.smithi047.stdout:(115/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.3 MB/s | 54 kB 00:00 2023-06-22T17:00:54.174 INFO:teuthology.orchestra.run.smithi047.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 25 MB/s | 713 kB 00:00 2023-06-22T17:00:54.225 INFO:teuthology.orchestra.run.smithi047.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 51 MB/s | 4.7 MB 00:00 2023-06-22T17:00:54.291 INFO:teuthology.orchestra.run.smithi047.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 30 MB/s | 4.6 MB 00:00 2023-06-22T17:00:55.060 INFO:teuthology.orchestra.run.smithi047.stdout:(119/120): librados2-17.2.6-719.g42ddc46e.el8.x 3.9 MB/s | 3.5 MB 00:00 2023-06-22T17:00:55.331 INFO:teuthology.orchestra.run.smithi047.stdout:(120/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 3.2 MB/s | 3.6 MB 00:01 2023-06-22T17:00:55.334 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2023-06-22T17:00:55.334 INFO:teuthology.orchestra.run.smithi047.stdout:Total 13 MB/s | 201 MB 00:15 2023-06-22T17:00:55.390 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2023-06-22T17:00:55.516 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2023-06-22T17:00:55.517 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2023-06-22T17:00:57.197 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2023-06-22T17:00:57.202 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2023-06-22T17:01:02.814 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2023-06-22T17:01:03.095 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-22T17:01:03.310 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-22T17:01:03.416 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-22T17:01:03.627 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-22T17:01:04.035 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-22T17:01:04.235 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T17:01:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T17:01:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-22T17:01:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-22T17:01:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T17:01:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T17:01:06.200 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T17:01:06.597 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-06-22T17:01:07.281 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-22T17:01:07.476 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T17:01:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T17:01:07.823 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-22T17:01:08.009 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-22T17:01:08.218 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T17:01:08.337 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T17:01:08.634 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-22T17:01:08.739 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-22T17:01:08.925 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-06-22T17:01:09.229 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-06-22T17:01:09.467 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-06-22T17:01:09.650 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-06-22T17:01:09.818 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-22T17:01:09.949 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-22T17:01:10.142 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-22T17:01:10.715 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-22T17:01:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-22T17:01:11.518 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-22T17:01:11.725 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-06-22T17:01:12.511 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-06-22T17:01:12.906 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-06-22T17:01:13.182 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-06-22T17:01:13.577 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-22T17:01:13.775 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-22T17:01:13.905 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-22T17:01:14.187 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-22T17:01:14.426 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-06-22T17:01:14.715 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-06-22T17:01:14.942 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-22T17:01:17.152 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-22T17:01:17.553 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-06-22T17:01:18.667 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-06-22T17:01:19.041 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-06-22T17:01:19.349 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-06-22T17:01:19.602 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-22T17:01:19.854 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-22T17:01:20.049 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-22T17:01:20.239 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-22T17:01:20.539 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-22T17:01:20.782 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-06-22T17:01:20.940 INFO:teuthology.orchestra.run.smithi202.stdout:Extra Packages for Enterprise Linux 261 kB/s | 16 MB 01:01 2023-06-22T17:01:20.992 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-06-22T17:01:21.230 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-06-22T17:01:21.424 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-06-22T17:01:21.608 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-22T17:01:21.715 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-22T17:01:21.884 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-06-22T17:01:22.069 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-06-22T17:01:22.296 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-06-22T17:01:22.463 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-06-22T17:01:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-06-22T17:01:22.819 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-06-22T17:01:23.098 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-06-22T17:01:23.294 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-06-22T17:01:23.488 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-06-22T17:01:23.823 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-06-22T17:01:24.109 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-06-22T17:01:24.163 INFO:teuthology.orchestra.run.smithi202.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-06-22T17:01:24.398 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-06-22T17:01:24.541 INFO:teuthology.orchestra.run.smithi202.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 129 MB/s | 38 MB 00:00 2023-06-22T17:01:24.686 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-06-22T17:01:24.817 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-22T17:01:25.025 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-22T17:01:25.326 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-22T17:01:25.567 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-22T17:01:25.783 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-22T17:01:25.959 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-22T17:01:26.222 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-22T17:01:26.459 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-22T17:01:26.693 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-22T17:01:26.941 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-22T17:01:27.145 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-22T17:01:27.332 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-22T17:01:27.647 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-22T17:01:27.897 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-22T17:01:28.148 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-22T17:01:28.481 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-22T17:01:28.733 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-22T17:01:28.969 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-22T17:01:29.250 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-22T17:01:30.061 INFO:teuthology.orchestra.run.smithi202.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 124 MB/s | 33 MB 00:00 2023-06-22T17:01:30.493 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-22T17:01:30.824 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-22T17:01:31.077 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-22T17:01:31.780 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-22T17:01:32.197 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-22T17:01:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-22T17:01:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-22T17:01:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-22T17:01:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-22T17:01:37.994 INFO:teuthology.orchestra.run.smithi202.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-22T17:01:37.995 INFO:teuthology.orchestra.run.smithi202.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-22T17:01:38.005 INFO:teuthology.orchestra.run.smithi202.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-22T17:01:38.006 INFO:teuthology.orchestra.run.smithi202.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-22T17:01:38.007 INFO:teuthology.orchestra.run.smithi202.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-22T17:01:38.008 INFO:teuthology.orchestra.run.smithi202.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-22T17:01:38.010 INFO:teuthology.orchestra.run.smithi202.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-22T17:01:38.012 INFO:teuthology.orchestra.run.smithi202.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-22T17:01:38.013 INFO:teuthology.orchestra.run.smithi202.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-22T17:01:38.498 INFO:teuthology.orchestra.run.smithi202.stdout:Dependencies resolved. 2023-06-22T17:01:38.509 INFO:teuthology.orchestra.run.smithi202.stdout:====================================================================================================================== 2023-06-22T17:01:38.509 INFO:teuthology.orchestra.run.smithi202.stdout: Package Arch Version Repository Size 2023-06-22T17:01:38.509 INFO:teuthology.orchestra.run.smithi202.stdout:====================================================================================================================== 2023-06-22T17:01:38.510 INFO:teuthology.orchestra.run.smithi202.stdout:Installing: 2023-06-22T17:01:38.510 INFO:teuthology.orchestra.run.smithi202.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-22T17:01:38.510 INFO:teuthology.orchestra.run.smithi202.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-22T17:01:38.510 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-22T17:01:38.510 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-22T17:01:38.510 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-22T17:01:38.510 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-22T17:01:38.511 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-22T17:01:38.511 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-22T17:01:38.511 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-22T17:01:38.511 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-22T17:01:38.511 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-22T17:01:38.511 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-22T17:01:38.511 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-22T17:01:38.512 INFO:teuthology.orchestra.run.smithi202.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-22T17:01:38.512 INFO:teuthology.orchestra.run.smithi202.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-22T17:01:38.512 INFO:teuthology.orchestra.run.smithi202.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-22T17:01:38.512 INFO:teuthology.orchestra.run.smithi202.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-22T17:01:38.512 INFO:teuthology.orchestra.run.smithi202.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-22T17:01:38.513 INFO:teuthology.orchestra.run.smithi202.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-22T17:01:38.513 INFO:teuthology.orchestra.run.smithi202.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-22T17:01:38.513 INFO:teuthology.orchestra.run.smithi202.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-22T17:01:38.513 INFO:teuthology.orchestra.run.smithi202.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-22T17:01:38.513 INFO:teuthology.orchestra.run.smithi202.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-22T17:01:38.514 INFO:teuthology.orchestra.run.smithi202.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-22T17:01:38.514 INFO:teuthology.orchestra.run.smithi202.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-22T17:01:38.514 INFO:teuthology.orchestra.run.smithi202.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-22T17:01:38.514 INFO:teuthology.orchestra.run.smithi202.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-22T17:01:38.514 INFO:teuthology.orchestra.run.smithi202.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-22T17:01:38.514 INFO:teuthology.orchestra.run.smithi202.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-22T17:01:38.514 INFO:teuthology.orchestra.run.smithi202.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-22T17:01:38.515 INFO:teuthology.orchestra.run.smithi202.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-22T17:01:38.515 INFO:teuthology.orchestra.run.smithi202.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-22T17:01:38.515 INFO:teuthology.orchestra.run.smithi202.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-22T17:01:38.515 INFO:teuthology.orchestra.run.smithi202.stdout:Upgrading: 2023-06-22T17:01:38.515 INFO:teuthology.orchestra.run.smithi202.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-22T17:01:38.515 INFO:teuthology.orchestra.run.smithi202.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-22T17:01:38.516 INFO:teuthology.orchestra.run.smithi202.stdout:Installing dependencies: 2023-06-22T17:01:38.517 INFO:teuthology.orchestra.run.smithi202.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-22T17:01:38.517 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-22T17:01:38.517 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-22T17:01:38.517 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-22T17:01:38.517 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-22T17:01:38.517 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-22T17:01:38.517 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-22T17:01:38.518 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-22T17:01:38.518 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-22T17:01:38.518 INFO:teuthology.orchestra.run.smithi202.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-22T17:01:38.518 INFO:teuthology.orchestra.run.smithi202.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-22T17:01:38.518 INFO:teuthology.orchestra.run.smithi202.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-22T17:01:38.518 INFO:teuthology.orchestra.run.smithi202.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-22T17:01:38.518 INFO:teuthology.orchestra.run.smithi202.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-22T17:01:38.519 INFO:teuthology.orchestra.run.smithi202.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-22T17:01:38.519 INFO:teuthology.orchestra.run.smithi202.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-22T17:01:38.519 INFO:teuthology.orchestra.run.smithi202.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-22T17:01:38.519 INFO:teuthology.orchestra.run.smithi202.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-22T17:01:38.519 INFO:teuthology.orchestra.run.smithi202.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-22T17:01:38.519 INFO:teuthology.orchestra.run.smithi202.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-22T17:01:38.519 INFO:teuthology.orchestra.run.smithi202.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-22T17:01:38.520 INFO:teuthology.orchestra.run.smithi202.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-22T17:01:38.520 INFO:teuthology.orchestra.run.smithi202.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-22T17:01:38.520 INFO:teuthology.orchestra.run.smithi202.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-22T17:01:38.520 INFO:teuthology.orchestra.run.smithi202.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-22T17:01:38.520 INFO:teuthology.orchestra.run.smithi202.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-22T17:01:38.520 INFO:teuthology.orchestra.run.smithi202.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-22T17:01:38.520 INFO:teuthology.orchestra.run.smithi202.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-22T17:01:38.520 INFO:teuthology.orchestra.run.smithi202.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-22T17:01:38.521 INFO:teuthology.orchestra.run.smithi202.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-22T17:01:38.521 INFO:teuthology.orchestra.run.smithi202.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-22T17:01:38.521 INFO:teuthology.orchestra.run.smithi202.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-22T17:01:38.521 INFO:teuthology.orchestra.run.smithi202.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-22T17:01:38.521 INFO:teuthology.orchestra.run.smithi202.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-22T17:01:38.521 INFO:teuthology.orchestra.run.smithi202.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-22T17:01:38.521 INFO:teuthology.orchestra.run.smithi202.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-22T17:01:38.522 INFO:teuthology.orchestra.run.smithi202.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-22T17:01:38.522 INFO:teuthology.orchestra.run.smithi202.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-22T17:01:38.522 INFO:teuthology.orchestra.run.smithi202.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-22T17:01:38.522 INFO:teuthology.orchestra.run.smithi202.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-22T17:01:38.522 INFO:teuthology.orchestra.run.smithi202.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-22T17:01:38.522 INFO:teuthology.orchestra.run.smithi202.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-22T17:01:38.522 INFO:teuthology.orchestra.run.smithi202.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-22T17:01:38.523 INFO:teuthology.orchestra.run.smithi202.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-22T17:01:38.523 INFO:teuthology.orchestra.run.smithi202.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-22T17:01:38.523 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-22T17:01:38.523 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-22T17:01:38.523 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-22T17:01:38.523 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-22T17:01:38.523 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-22T17:01:38.523 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-22T17:01:38.524 INFO:teuthology.orchestra.run.smithi202.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-22T17:01:38.524 INFO:teuthology.orchestra.run.smithi202.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-22T17:01:38.524 INFO:teuthology.orchestra.run.smithi202.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-22T17:01:38.524 INFO:teuthology.orchestra.run.smithi202.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-22T17:01:38.524 INFO:teuthology.orchestra.run.smithi202.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-22T17:01:38.524 INFO:teuthology.orchestra.run.smithi202.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-22T17:01:38.524 INFO:teuthology.orchestra.run.smithi202.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-22T17:01:38.525 INFO:teuthology.orchestra.run.smithi202.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-22T17:01:38.525 INFO:teuthology.orchestra.run.smithi202.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-22T17:01:38.525 INFO:teuthology.orchestra.run.smithi202.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-22T17:01:38.525 INFO:teuthology.orchestra.run.smithi202.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-22T17:01:38.525 INFO:teuthology.orchestra.run.smithi202.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-22T17:01:38.525 INFO:teuthology.orchestra.run.smithi202.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-22T17:01:38.525 INFO:teuthology.orchestra.run.smithi202.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-22T17:01:38.525 INFO:teuthology.orchestra.run.smithi202.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-22T17:01:38.526 INFO:teuthology.orchestra.run.smithi202.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-22T17:01:38.526 INFO:teuthology.orchestra.run.smithi202.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-22T17:01:38.526 INFO:teuthology.orchestra.run.smithi202.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-22T17:01:38.526 INFO:teuthology.orchestra.run.smithi202.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-22T17:01:38.526 INFO:teuthology.orchestra.run.smithi202.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-22T17:01:38.526 INFO:teuthology.orchestra.run.smithi202.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-22T17:01:38.526 INFO:teuthology.orchestra.run.smithi202.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-22T17:01:38.526 INFO:teuthology.orchestra.run.smithi202.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-22T17:01:38.527 INFO:teuthology.orchestra.run.smithi202.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-22T17:01:38.527 INFO:teuthology.orchestra.run.smithi202.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-22T17:01:38.527 INFO:teuthology.orchestra.run.smithi202.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-22T17:01:38.527 INFO:teuthology.orchestra.run.smithi202.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-22T17:01:38.527 INFO:teuthology.orchestra.run.smithi202.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-22T17:01:38.527 INFO:teuthology.orchestra.run.smithi202.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-22T17:01:38.527 INFO:teuthology.orchestra.run.smithi202.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-22T17:01:38.528 INFO:teuthology.orchestra.run.smithi202.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-22T17:01:38.528 INFO:teuthology.orchestra.run.smithi202.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-22T17:01:38.528 INFO:teuthology.orchestra.run.smithi202.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-22T17:01:38.528 INFO:teuthology.orchestra.run.smithi202.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-22T17:01:38.528 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:01:38.528 INFO:teuthology.orchestra.run.smithi202.stdout:Transaction Summary 2023-06-22T17:01:38.528 INFO:teuthology.orchestra.run.smithi202.stdout:====================================================================================================================== 2023-06-22T17:01:38.529 INFO:teuthology.orchestra.run.smithi202.stdout:Install 118 Packages 2023-06-22T17:01:38.529 INFO:teuthology.orchestra.run.smithi202.stdout:Upgrade 2 Packages 2023-06-22T17:01:38.529 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:01:38.529 INFO:teuthology.orchestra.run.smithi202.stdout:Total download size: 201 M 2023-06-22T17:01:38.530 INFO:teuthology.orchestra.run.smithi202.stdout:Downloading Packages: 2023-06-22T17:01:38.644 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T17:01:38.845 INFO:teuthology.orchestra.run.smithi202.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 42 kB/s | 6.4 kB 00:00 2023-06-22T17:01:38.906 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T17:01:39.453 INFO:teuthology.orchestra.run.smithi202.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 1.4 MB/s | 873 kB 00:00 2023-06-22T17:01:39.533 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T17:01:39.604 INFO:teuthology.orchestra.run.smithi202.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 1.1 MB/s | 162 kB 00:00 2023-06-22T17:01:39.814 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-22T17:01:39.915 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-22T17:01:39.924 INFO:teuthology.orchestra.run.smithi202.stdout:(4/120): ceph-common-17.2.6-719.g42ddc46e.el8.x 19 MB/s | 23 MB 00:01 2023-06-22T17:01:40.011 INFO:teuthology.orchestra.run.smithi202.stdout:(5/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 18 MB/s | 1.6 MB 00:00 2023-06-22T17:01:40.071 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-22T17:01:40.213 INFO:teuthology.orchestra.run.smithi202.stdout:(6/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 19 MB/s | 3.9 MB 00:00 2023-06-22T17:01:40.312 INFO:teuthology.orchestra.run.smithi202.stdout:(7/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 3.2 MB/s | 5.2 MB 00:01 2023-06-22T17:01:40.419 INFO:teuthology.orchestra.run.smithi202.stdout:(8/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 2.8 MB/s | 2.3 MB 00:00 2023-06-22T17:01:40.504 INFO:teuthology.orchestra.run.smithi202.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 286 kB/s | 24 kB 00:00 2023-06-22T17:01:41.167 INFO:teuthology.orchestra.run.smithi202.stdout:(10/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86 18 MB/s | 17 MB 00:00 2023-06-22T17:01:41.218 INFO:teuthology.orchestra.run.smithi202.stdout:(11/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 4.9 MB/s | 255 kB 00:00 2023-06-22T17:01:41.253 INFO:teuthology.orchestra.run.smithi202.stdout:(12/120): libcephfs-devel-17.2.6-719.g42ddc46e. 714 kB/s | 25 kB 00:00 2023-06-22T17:01:41.310 INFO:teuthology.orchestra.run.smithi202.stdout:(13/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 14 MB/s | 794 kB 00:00 2023-06-22T17:01:41.358 INFO:teuthology.orchestra.run.smithi202.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 3.5 MB/s | 174 kB 00:00 2023-06-22T17:01:41.400 INFO:teuthology.orchestra.run.smithi202.stdout:(15/120): librados-devel-17.2.6-719.g42ddc46e.e 3.1 MB/s | 130 kB 00:00 2023-06-22T17:01:41.507 INFO:teuthology.orchestra.run.smithi202.stdout:(16/120): libradosstriper1-17.2.6-719.g42ddc46e 4.6 MB/s | 506 kB 00:00 2023-06-22T17:01:42.150 INFO:teuthology.orchestra.run.smithi202.stdout:(17/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8 9.1 MB/s | 17 MB 00:01 2023-06-22T17:01:42.296 INFO:teuthology.orchestra.run.smithi202.stdout:(18/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 7.0 MB/s | 5.5 MB 00:00 2023-06-22T17:01:42.374 INFO:teuthology.orchestra.run.smithi202.stdout:(19/120): python3-ceph-argparse-17.2.6-719.g42d 199 kB/s | 45 kB 00:00 2023-06-22T17:01:42.432 INFO:teuthology.orchestra.run.smithi202.stdout:(20/120): python3-ceph-common-17.2.6-719.g42ddc 849 kB/s | 115 kB 00:00 2023-06-22T17:01:42.434 INFO:teuthology.orchestra.run.smithi202.stdout:(21/120): python3-cephfs-17.2.6-719.g42ddc46e.e 3.1 MB/s | 190 kB 00:00 2023-06-22T17:01:42.484 INFO:teuthology.orchestra.run.smithi202.stdout:(22/120): python3-rados-17.2.6-719.g42ddc46e.el 7.0 MB/s | 371 kB 00:00 2023-06-22T17:01:42.497 INFO:teuthology.orchestra.run.smithi202.stdout:(23/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 5.6 MB/s | 357 kB 00:00 2023-06-22T17:01:42.506 INFO:teuthology.orchestra.run.smithi202.stdout:(24/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 4.9 MB/s | 107 kB 00:00 2023-06-22T17:01:42.534 INFO:teuthology.orchestra.run.smithi202.stdout:(25/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 2.4 MB/s | 91 kB 00:00 2023-06-22T17:01:42.585 INFO:teuthology.orchestra.run.smithi202.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 3.3 MB/s | 173 kB 00:00 2023-06-22T17:01:42.631 INFO:teuthology.orchestra.run.smithi202.stdout:(27/120): ceph-grafana-dashboards-17.2.6-719.g4 510 kB/s | 23 kB 00:00 2023-06-22T17:01:42.679 INFO:teuthology.orchestra.run.smithi202.stdout:(28/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 2.5 MB/s | 121 kB 00:00 2023-06-22T17:01:42.717 INFO:teuthology.orchestra.run.smithi202.stdout:(29/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 16 MB/s | 3.5 MB 00:00 2023-06-22T17:01:42.857 INFO:teuthology.orchestra.run.smithi202.stdout:(30/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 9.5 MB/s | 1.7 MB 00:00 2023-06-22T17:01:42.912 INFO:teuthology.orchestra.run.smithi202.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 4.3 MB/s | 238 kB 00:00 2023-06-22T17:01:42.958 INFO:teuthology.orchestra.run.smithi202.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 1.0 MB/s | 47 kB 00:00 2023-06-22T17:01:43.005 INFO:teuthology.orchestra.run.smithi202.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 311 kB/s | 14 kB 00:00 2023-06-22T17:01:43.061 INFO:teuthology.orchestra.run.smithi202.stdout:(34/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 4.6 MB/s | 264 kB 00:00 2023-06-22T17:01:43.112 INFO:teuthology.orchestra.run.smithi202.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 1.6 MB/s | 80 kB 00:00 2023-06-22T17:01:43.159 INFO:teuthology.orchestra.run.smithi202.stdout:(36/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 377 kB/s | 18 kB 00:00 2023-06-22T17:01:43.215 INFO:teuthology.orchestra.run.smithi202.stdout:(37/120): ceph-mgr-diskprediction-local-17.2.6- 15 MB/s | 7.4 MB 00:00 2023-06-22T17:01:43.423 INFO:teuthology.orchestra.run.smithi202.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 284 kB/s | 59 kB 00:00 2023-06-22T17:01:43.466 INFO:teuthology.orchestra.run.smithi202.stdout:(39/120): gperftools-libs-2.7-9.el8.x86_64.rpm 998 kB/s | 306 kB 00:00 2023-06-22T17:01:43.546 INFO:teuthology.orchestra.run.smithi202.stdout:(40/120): libunwind-1.3.1-3.el8.x86_64.rpm 612 kB/s | 75 kB 00:00 2023-06-22T17:01:43.634 INFO:teuthology.orchestra.run.smithi202.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 496 kB/s | 44 kB 00:00 2023-06-22T17:01:43.892 INFO:teuthology.orchestra.run.smithi202.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 717 kB/s | 185 kB 00:00 2023-06-22T17:01:44.092 INFO:teuthology.orchestra.run.smithi202.stdout:(43/120): python3-asyncssh-2.7.0-2.el8.noarch.r 760 kB/s | 476 kB 00:00 2023-06-22T17:01:44.233 INFO:teuthology.orchestra.run.smithi202.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 97 kB/s | 33 kB 00:00 2023-06-22T17:01:44.269 INFO:teuthology.orchestra.run.smithi202.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 92 kB/s | 16 kB 00:00 2023-06-22T17:01:44.565 INFO:teuthology.orchestra.run.smithi202.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 520 kB/s | 173 kB 00:00 2023-06-22T17:01:44.796 INFO:teuthology.orchestra.run.smithi202.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 351 kB/s | 81 kB 00:00 2023-06-22T17:01:44.804 INFO:teuthology.orchestra.run.smithi202.stdout:(48/120): python3-cherrypy-18.4.0-1.el8.noarch. 720 kB/s | 384 kB 00:00 2023-06-22T17:01:44.990 INFO:teuthology.orchestra.run.smithi202.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 94 kB/s | 18 kB 00:00 2023-06-22T17:01:45.020 INFO:teuthology.orchestra.run.smithi202.stdout:(50/120): python3-jaraco-6.2-6.el8.noarch.rpm 50 kB/s | 11 kB 00:00 2023-06-22T17:01:45.184 INFO:teuthology.orchestra.run.smithi202.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 297 kB/s | 49 kB 00:00 2023-06-22T17:01:45.357 INFO:teuthology.orchestra.run.smithi202.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 339 kB/s | 59 kB 00:00 2023-06-22T17:01:45.923 INFO:teuthology.orchestra.run.smithi202.stdout:(53/120): ceph-test-17.2.6-719.g42ddc46e.el8.x8 9.5 MB/s | 52 MB 00:05 2023-06-22T17:01:45.924 INFO:teuthology.orchestra.run.smithi202.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 106 kB/s | 60 kB 00:00 2023-06-22T17:01:46.066 INFO:teuthology.orchestra.run.smithi202.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 110 kB/s | 16 kB 00:00 2023-06-22T17:01:46.157 INFO:teuthology.orchestra.run.smithi202.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 373 kB/s | 34 kB 00:00 2023-06-22T17:01:46.285 INFO:teuthology.orchestra.run.smithi202.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-06-22T17:01:46.330 INFO:teuthology.orchestra.run.smithi202.stdout:(58/120): python3-rsa-4.8-1.el8.noarch.rpm 1.3 MB/s | 61 kB 00:00 2023-06-22T17:01:46.371 INFO:teuthology.orchestra.run.smithi202.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 457 kB/s | 19 kB 00:00 2023-06-22T17:01:46.427 INFO:teuthology.orchestra.run.smithi202.stdout:(60/120): python3-singledispatch-3.4.0.3-18.el8 426 kB/s | 24 kB 00:00 2023-06-22T17:01:46.484 INFO:teuthology.orchestra.run.smithi202.stdout:(61/120): python3-tempora-1.14.1-5.el8.noarch.r 513 kB/s | 29 kB 00:00 2023-06-22T17:01:46.536 INFO:teuthology.orchestra.run.smithi202.stdout:(62/120): python3-trustme-0.6.0-4.el8.noarch.rp 541 kB/s | 27 kB 00:00 2023-06-22T17:01:46.596 INFO:teuthology.orchestra.run.smithi202.stdout:(63/120): python3-waitress-1.4.3-1.el8.noarch.r 4.2 MB/s | 255 kB 00:00 2023-06-22T17:01:46.982 INFO:teuthology.orchestra.run.smithi202.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 650 kB/s | 251 kB 00:00 2023-06-22T17:01:47.032 INFO:teuthology.orchestra.run.smithi202.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-06-22T17:01:47.310 INFO:teuthology.orchestra.run.smithi202.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 308 kB/s | 85 kB 00:00 2023-06-22T17:01:47.351 INFO:teuthology.orchestra.run.smithi202.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 560 kB/s | 23 kB 00:00 2023-06-22T17:01:47.457 INFO:teuthology.orchestra.run.smithi202.stdout:(68/120): python3-pecan-1.3.2-9.el8.noarch.rpm 184 kB/s | 283 kB 00:01 2023-06-22T17:01:47.462 INFO:teuthology.orchestra.run.smithi202.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 14 MB/s | 70 kB 00:00 2023-06-22T17:01:47.493 INFO:teuthology.orchestra.run.smithi202.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 600 kB/s | 19 kB 00:00 2023-06-22T17:01:47.509 INFO:teuthology.orchestra.run.smithi202.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.7 MB/s | 43 kB 00:00 2023-06-22T17:01:47.525 INFO:teuthology.orchestra.run.smithi202.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 9.9 MB/s | 155 kB 00:00 2023-06-22T17:01:47.541 INFO:teuthology.orchestra.run.smithi202.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-06-22T17:01:47.556 INFO:teuthology.orchestra.run.smithi202.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.3 MB/s | 48 kB 00:00 2023-06-22T17:01:47.571 INFO:teuthology.orchestra.run.smithi202.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 15 MB/s | 229 kB 00:00 2023-06-22T17:01:47.583 INFO:teuthology.orchestra.run.smithi202.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 3.3 MB/s | 43 kB 00:00 2023-06-22T17:01:47.624 INFO:teuthology.orchestra.run.smithi202.stdout:(77/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 57 MB/s | 2.3 MB 00:00 2023-06-22T17:01:47.637 INFO:teuthology.orchestra.run.smithi202.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.4 MB/s | 18 kB 00:00 2023-06-22T17:01:47.655 INFO:teuthology.orchestra.run.smithi202.stdout:(79/120): pcre2-devel-10.32-2.el8.x86_64.rpm 32 MB/s | 605 kB 00:00 2023-06-22T17:01:47.669 INFO:teuthology.orchestra.run.smithi202.stdout:(80/120): fuse-2.9.7-12.el8.x86_64.rpm 6.1 MB/s | 83 kB 00:00 2023-06-22T17:01:47.681 INFO:teuthology.orchestra.run.smithi202.stdout:(81/120): elfutils-libelf-devel-0.182-3.el8.x86 4.6 MB/s | 59 kB 00:00 2023-06-22T17:01:47.695 INFO:teuthology.orchestra.run.smithi202.stdout:(82/120): libquadmath-8.4.1-1.el8.x86_64.rpm 12 MB/s | 169 kB 00:00 2023-06-22T17:01:47.708 INFO:teuthology.orchestra.run.smithi202.stdout:(83/120): libcom_err-devel-1.45.6-1.el8.x86_64. 3.0 MB/s | 38 kB 00:00 2023-06-22T17:01:47.720 INFO:teuthology.orchestra.run.smithi202.stdout:(84/120): libsepol-devel-2.9-2.el8.x86_64.rpm 7.1 MB/s | 86 kB 00:00 2023-06-22T17:01:47.732 INFO:teuthology.orchestra.run.smithi202.stdout:(85/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.9 MB/s | 58 kB 00:00 2023-06-22T17:01:47.743 INFO:teuthology.orchestra.run.smithi202.stdout:(86/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 4.1 MB/s | 47 kB 00:00 2023-06-22T17:01:47.760 INFO:teuthology.orchestra.run.smithi202.stdout:(87/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 32 MB/s | 559 kB 00:00 2023-06-22T17:01:47.780 INFO:teuthology.orchestra.run.smithi202.stdout:(88/120): libgfortran-8.4.1-1.el8.x86_64.rpm 32 MB/s | 639 kB 00:00 2023-06-22T17:01:47.792 INFO:teuthology.orchestra.run.smithi202.stdout:(89/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 5.8 MB/s | 70 kB 00:00 2023-06-22T17:01:47.805 INFO:teuthology.orchestra.run.smithi202.stdout:(90/120): libselinux-devel-2.9-5.el8.x86_64.rpm 15 MB/s | 200 kB 00:00 2023-06-22T17:01:47.819 INFO:teuthology.orchestra.run.smithi202.stdout:(91/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 7.2 MB/s | 97 kB 00:00 2023-06-22T17:01:48.058 INFO:teuthology.orchestra.run.smithi202.stdout:(92/120): python3-scipy-1.0.0-20.module+el8.1.0 61 MB/s | 15 MB 00:00 2023-06-22T17:01:48.061 INFO:teuthology.orchestra.run.smithi202.stdout:(93/120): python3-kubernetes-11.0.0-6.el8.noarc 389 kB/s | 1.2 MB 00:03 2023-06-22T17:01:48.067 INFO:teuthology.orchestra.run.smithi202.stdout:(94/120): thrift-0.13.0-2.el8.x86_64.rpm 2.4 MB/s | 1.7 MB 00:00 2023-06-22T17:01:48.081 INFO:teuthology.orchestra.run.smithi202.stdout:(95/120): python3-prettytable-0.7.2-14.el8.noar 1.9 MB/s | 44 kB 00:00 2023-06-22T17:01:48.089 INFO:teuthology.orchestra.run.smithi202.stdout:(96/120): python3-jsonpatch-1.21-2.el8.noarch.r 967 kB/s | 27 kB 00:00 2023-06-22T17:01:48.097 INFO:teuthology.orchestra.run.smithi202.stdout:(97/120): jq-1.5-12.el8.x86_64.rpm 5.3 MB/s | 161 kB 00:00 2023-06-22T17:01:48.100 INFO:teuthology.orchestra.run.smithi202.stdout:(98/120): flex-2.6.1-9.el8.x86_64.rpm 17 MB/s | 320 kB 00:00 2023-06-22T17:01:48.108 INFO:teuthology.orchestra.run.smithi202.stdout:(99/120): python3-werkzeug-0.12.2-4.el8.noarch. 24 MB/s | 457 kB 00:00 2023-06-22T17:01:48.114 INFO:teuthology.orchestra.run.smithi202.stdout:(100/120): python3-markupsafe-0.23-19.el8.x86_6 2.7 MB/s | 39 kB 00:00 2023-06-22T17:01:48.134 INFO:teuthology.orchestra.run.smithi202.stdout:(101/120): bison-3.0.4-10.el8.x86_64.rpm 26 MB/s | 688 kB 00:00 2023-06-22T17:01:48.135 INFO:teuthology.orchestra.run.smithi202.stdout:(102/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 12 MB/s | 259 kB 00:00 2023-06-22T17:01:48.152 INFO:teuthology.orchestra.run.smithi202.stdout:(103/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 7.2 MB/s | 126 kB 00:00 2023-06-22T17:01:48.154 INFO:teuthology.orchestra.run.smithi202.stdout:(104/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 11 MB/s | 225 kB 00:00 2023-06-22T17:01:48.171 INFO:teuthology.orchestra.run.smithi202.stdout:(105/120): python3-babel-2.5.1-5.el8.noarch.rpm 65 MB/s | 4.8 MB 00:00 2023-06-22T17:01:48.174 INFO:teuthology.orchestra.run.smithi202.stdout:(106/120): oniguruma-6.8.2-2.el8.x86_64.rpm 9.5 MB/s | 187 kB 00:00 2023-06-22T17:01:48.191 INFO:teuthology.orchestra.run.smithi202.stdout:(107/120): python3-jinja2-2.10.1-2.el8_0.noarch 27 MB/s | 538 kB 00:00 2023-06-22T17:01:48.192 INFO:teuthology.orchestra.run.smithi202.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 16 MB/s | 302 kB 00:00 2023-06-22T17:01:48.206 INFO:teuthology.orchestra.run.smithi202.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-06-22T17:01:48.207 INFO:teuthology.orchestra.run.smithi202.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 7.2 MB/s | 110 kB 00:00 2023-06-22T17:01:48.220 INFO:teuthology.orchestra.run.smithi202.stdout:(111/120): python3-numpy-1.14.3-9.el8.x86_64.rp 56 MB/s | 3.7 MB 00:00 2023-06-22T17:01:48.220 INFO:teuthology.orchestra.run.smithi202.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.5 MB/s | 20 kB 00:00 2023-06-22T17:01:48.224 INFO:teuthology.orchestra.run.smithi202.stdout:(113/120): librdkafka-0.11.4-1.el8.x86_64.rpm 19 MB/s | 353 kB 00:00 2023-06-22T17:01:48.233 INFO:teuthology.orchestra.run.smithi202.stdout:(114/120): python3-pytz-2017.2-9.el8.noarch.rpm 4.1 MB/s | 54 kB 00:00 2023-06-22T17:01:48.241 INFO:teuthology.orchestra.run.smithi202.stdout:(115/120): libtool-2.4.6-25.el8.x86_64.rpm 33 MB/s | 709 kB 00:00 2023-06-22T17:01:48.265 INFO:teuthology.orchestra.run.smithi202.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 29 MB/s | 713 kB 00:00 2023-06-22T17:01:48.305 INFO:teuthology.orchestra.run.smithi202.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 58 MB/s | 4.7 MB 00:00 2023-06-22T17:01:48.324 INFO:teuthology.orchestra.run.smithi202.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 51 MB/s | 4.6 MB 00:00 2023-06-22T17:01:48.715 INFO:teuthology.orchestra.run.smithi202.stdout:(119/120): librados2-17.2.6-719.g42ddc46e.el8.x 7.7 MB/s | 3.5 MB 00:00 2023-06-22T17:01:48.725 INFO:teuthology.orchestra.run.smithi202.stdout:(120/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 8.6 MB/s | 3.6 MB 00:00 2023-06-22T17:01:48.728 INFO:teuthology.orchestra.run.smithi202.stdout:-------------------------------------------------------------------------------- 2023-06-22T17:01:48.728 INFO:teuthology.orchestra.run.smithi202.stdout:Total 20 MB/s | 201 MB 00:10 2023-06-22T17:01:48.776 INFO:teuthology.orchestra.run.smithi202.stdout:Running transaction check 2023-06-22T17:01:48.889 INFO:teuthology.orchestra.run.smithi202.stdout:Transaction check succeeded. 2023-06-22T17:01:48.889 INFO:teuthology.orchestra.run.smithi202.stdout:Running transaction test 2023-06-22T17:01:50.581 INFO:teuthology.orchestra.run.smithi202.stdout:Transaction test succeeded. 2023-06-22T17:01:50.585 INFO:teuthology.orchestra.run.smithi202.stdout:Running transaction 2023-06-22T17:01:55.235 INFO:teuthology.orchestra.run.smithi202.stdout: Preparing : 1/1 2023-06-22T17:01:55.461 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-22T17:01:55.656 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-22T17:01:55.787 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-22T17:01:55.997 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-22T17:01:56.371 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-22T17:01:56.622 INFO:teuthology.orchestra.run.smithi202.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T17:01:57.007 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T17:01:57.189 INFO:teuthology.orchestra.run.smithi202.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-22T17:01:57.327 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-22T17:01:57.564 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T17:01:57.665 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T17:01:57.864 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T17:01:58.194 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-06-22T17:01:58.689 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-22T17:01:58.689 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2023-06-22T17:01:58.689 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2023-06-22T17:01:58.689 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /dev 2023-06-22T17:01:58.690 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /run 2023-06-22T17:01:58.690 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2023-06-22T17:01:58.690 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2023-06-22T17:01:58.690 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2023-06-22T17:01:58.690 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2023-06-22T17:01:58.690 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /dev 2023-06-22T17:01:58.691 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:01:58.865 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-22T17:01:59.022 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T17:01:59.053 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-22T17:01:59.137 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T17:01:59.401 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-22T17:01:59.596 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-22T17:01:59.780 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T17:01:59.880 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T17:02:00.030 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-22T17:02:00.091 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-22T17:02:00.284 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-06-22T17:02:00.535 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-06-22T17:02:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-22T17:02:00.762 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-06-22T17:02:00.828 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-22T17:02:00.944 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-06-22T17:02:01.063 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-22T17:02:01.080 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-22T17:02:01.197 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-22T17:02:01.203 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-22T17:02:01.354 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-22T17:02:01.618 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-22T17:02:01.794 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-22T17:02:01.812 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-22T17:02:01.911 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-22T17:02:02.071 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-22T17:02:02.602 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-22T17:02:02.779 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-06-22T17:02:03.528 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-06-22T17:02:03.783 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-06-22T17:02:04.055 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-06-22T17:02:04.299 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-22T17:02:04.469 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-22T17:02:04.624 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-22T17:02:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-22T17:02:04.838 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-22T17:02:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-22T17:02:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-22T17:02:05.085 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-06-22T17:02:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-22T17:02:05.345 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-06-22T17:02:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-22T17:02:05.595 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-22T17:02:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-22T17:02:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-22T17:02:06.927 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-22T17:02:07.102 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-22T17:02:07.205 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-22T17:02:07.608 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-22T17:02:07.664 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-22T17:02:07.835 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-22T17:02:07.873 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-22T17:02:08.011 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-06-22T17:02:08.060 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-22T17:02:08.072 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-22T17:02:09.365 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-06-22T17:02:09.711 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-06-22T17:02:09.740 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-22T17:02:09.983 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-06-22T17:02:09.993 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-22T17:02:10.086 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-22T17:02:10.243 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-22T17:02:10.260 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-22T17:02:10.305 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-22T17:02:10.423 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-22T17:02:10.485 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-22T17:02:10.642 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-22T17:02:10.800 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-22T17:02:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-22T17:02:11.068 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-22T17:02:11.110 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-22T17:02:11.352 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-06-22T17:02:11.561 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-06-22T17:02:11.760 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-06-22T17:02:11.965 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-06-22T17:02:12.103 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-22T17:02:12.179 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-22T17:02:12.337 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-06-22T17:02:12.497 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-06-22T17:02:12.713 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-06-22T17:02:12.891 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-06-22T17:02:13.062 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-06-22T17:02:13.264 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-06-22T17:02:13.399 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-06-22T17:02:13.586 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-06-22T17:02:13.773 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-06-22T17:02:14.091 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-06-22T17:02:14.384 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-06-22T17:02:14.667 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-06-22T17:02:14.948 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-06-22T17:02:15.111 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-22T17:02:15.306 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-22T17:02:15.540 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-22T17:02:15.761 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-22T17:02:15.968 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-22T17:02:16.136 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-22T17:02:16.314 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-22T17:02:16.368 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-22T17:02:16.504 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-22T17:02:16.714 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-22T17:02:16.963 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-22T17:02:17.222 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-22T17:02:17.392 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-22T17:02:17.399 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-22T17:02:17.668 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-22T17:02:17.809 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-06-22T17:02:17.898 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-22T17:02:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 110/122 2023-06-22T17:02:18.091 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-22T17:02:18.260 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-06-22T17:02:18.387 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-22T17:02:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-06-22T17:02:18.463 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-06-22T17:02:18.614 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-22T17:02:18.651 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-22T17:02:18.772 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-22T17:02:18.840 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-22T17:02:18.909 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-22T17:02:19.117 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-22T17:02:19.144 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-22T17:02:19.404 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-22T17:02:19.617 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-06-22T17:02:19.671 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-06-22T17:02:19.858 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-06-22T17:02:19.868 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-06-22T17:02:20.011 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-06-22T17:02:20.011 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-22T17:02:20.195 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-22T17:02:20.195 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T17:02:20.292 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-22T17:02:20.633 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-22T17:02:20.763 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T17:02:20.763 INFO:teuthology.orchestra.run.smithi047.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-22T17:02:20.763 INFO:teuthology.orchestra.run.smithi047.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-22T17:02:20.763 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:02:20.836 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-22T17:02:21.107 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-22T17:02:21.107 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-22T17:02:21.108 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-22T17:02:21.108 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T17:02:21.108 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-22T17:02:21.108 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T17:02:21.108 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T17:02:21.108 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-22T17:02:21.109 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-22T17:02:21.109 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T17:02:21.109 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-22T17:02:21.109 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-22T17:02:21.109 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T17:02:21.109 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-22T17:02:21.109 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-22T17:02:21.110 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-22T17:02:21.110 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-22T17:02:21.110 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-22T17:02:21.110 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-22T17:02:21.110 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-22T17:02:21.110 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-22T17:02:21.110 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-22T17:02:21.111 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-22T17:02:21.111 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-22T17:02:21.111 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-22T17:02:21.111 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-22T17:02:21.111 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-22T17:02:21.111 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-22T17:02:21.111 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-22T17:02:21.112 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-22T17:02:21.112 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-22T17:02:21.112 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-22T17:02:21.112 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-22T17:02:21.112 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-22T17:02:21.112 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-22T17:02:21.112 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-22T17:02:21.113 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-22T17:02:21.113 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-22T17:02:21.113 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-22T17:02:21.113 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-22T17:02:21.114 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-22T17:02:21.114 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-22T17:02:21.115 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-22T17:02:21.115 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-22T17:02:21.115 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-22T17:02:21.116 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-22T17:02:21.116 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-22T17:02:21.116 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-22T17:02:21.116 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-22T17:02:21.117 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-22T17:02:21.117 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-22T17:02:21.117 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-22T17:02:21.117 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-22T17:02:21.117 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-22T17:02:21.118 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-22T17:02:21.118 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-22T17:02:21.118 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-22T17:02:21.118 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-22T17:02:21.119 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-22T17:02:21.119 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-22T17:02:21.119 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-22T17:02:21.120 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-22T17:02:21.120 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-22T17:02:21.120 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-22T17:02:21.120 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-22T17:02:21.120 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-22T17:02:21.120 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-22T17:02:21.120 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-22T17:02:21.121 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-22T17:02:21.121 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-22T17:02:21.121 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-22T17:02:21.121 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-06-22T17:02:21.121 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-06-22T17:02:21.121 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-06-22T17:02:21.121 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-06-22T17:02:21.122 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-06-22T17:02:21.122 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-06-22T17:02:21.122 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-06-22T17:02:21.122 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-06-22T17:02:21.122 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-06-22T17:02:21.122 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-06-22T17:02:21.122 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-06-22T17:02:21.122 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-06-22T17:02:21.123 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-06-22T17:02:21.123 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-06-22T17:02:21.123 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-06-22T17:02:21.123 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-06-22T17:02:21.123 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-06-22T17:02:21.123 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-06-22T17:02:21.123 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-06-22T17:02:21.123 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-06-22T17:02:21.124 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-06-22T17:02:21.125 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-06-22T17:02:21.125 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-06-22T17:02:21.126 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-06-22T17:02:21.126 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-06-22T17:02:21.126 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-06-22T17:02:21.126 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-06-22T17:02:21.126 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-06-22T17:02:21.126 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-06-22T17:02:21.126 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-06-22T17:02:21.127 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-06-22T17:02:21.127 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-06-22T17:02:21.127 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-06-22T17:02:21.127 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-06-22T17:02:21.127 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-06-22T17:02:21.127 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-06-22T17:02:21.127 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-06-22T17:02:21.128 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-06-22T17:02:21.128 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-06-22T17:02:21.128 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-06-22T17:02:21.128 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-06-22T17:02:21.128 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-06-22T17:02:21.128 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-06-22T17:02:21.128 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-06-22T17:02:21.128 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-06-22T17:02:21.129 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-06-22T17:02:21.129 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-06-22T17:02:21.129 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-22T17:02:21.129 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-22T17:02:21.129 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-22T17:02:21.155 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-22T17:02:21.388 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-22T17:02:21.742 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-22T17:02:21.978 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-22T17:02:22.169 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-22T17:02:22.410 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-22T17:02:23.325 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T17:02:23.325 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:53 ago on Thu 22 Jun 2023 05:00:30 PM UTC. 2023-06-22T17:02:25.642 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2023-06-22T17:02:25.794 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:02:25.795 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2023-06-22T17:02:25.795 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.795 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.795 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2023-06-22T17:02:25.795 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.1-7.el8.noarch 2023-06-22T17:02:25.795 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.0.4-10.el8.x86_64 2023-06-22T17:02:25.796 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.796 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.796 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.796 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.796 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:02:25.796 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.796 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:02:25.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:02:25.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:02:25.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:02:25.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:02:25.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.797 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.798 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:02:25.798 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.798 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.798 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.798 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:02:25.798 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:02:25.798 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.799 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:02:25.799 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-22T17:02:25.799 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.1-9.el8.x86_64 2023-06-22T17:02:25.799 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-22T17:02:25.799 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-22T17:02:25.799 INFO:teuthology.orchestra.run.smithi047.stdout: jq-1.5-12.el8.x86_64 2023-06-22T17:02:25.799 INFO:teuthology.orchestra.run.smithi047.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-22T17:02:25.800 INFO:teuthology.orchestra.run.smithi047.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-22T17:02:25.800 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-22T17:02:25.800 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.800 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.800 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.800 INFO:teuthology.orchestra.run.smithi047.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-22T17:02:25.800 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-22T17:02:25.800 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-22T17:02:25.801 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-22T17:02:25.801 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-22T17:02:25.801 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.801 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.801 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-22T17:02:25.801 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.801 INFO:teuthology.orchestra.run.smithi047.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-22T17:02:25.802 INFO:teuthology.orchestra.run.smithi047.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-22T17:02:25.802 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-22T17:02:25.802 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-22T17:02:25.802 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-22T17:02:25.802 INFO:teuthology.orchestra.run.smithi047.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-22T17:02:25.802 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-22T17:02:25.802 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-22T17:02:25.802 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-22T17:02:25.803 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-22T17:02:25.803 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-22T17:02:25.803 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-22T17:02:25.803 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-22T17:02:25.803 INFO:teuthology.orchestra.run.smithi047.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-22T17:02:25.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-22T17:02:25.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-22T17:02:25.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-22T17:02:25.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-22T17:02:25.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-22T17:02:25.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-22T17:02:25.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-22T17:02:25.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-22T17:02:25.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-22T17:02:25.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-22T17:02:25.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-22T17:02:25.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-22T17:02:25.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-22T17:02:25.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-22T17:02:25.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-22T17:02:25.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-22T17:02:25.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-22T17:02:25.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-22T17:02:25.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-22T17:02:25.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-22T17:02:25.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-22T17:02:25.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-22T17:02:25.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-22T17:02:25.807 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-22T17:02:25.807 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-22T17:02:25.807 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-2.6-1.el8.noarch 2023-06-22T17:02:25.807 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-22T17:02:25.807 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-22T17:02:25.807 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-22T17:02:25.807 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-22T17:02:25.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-22T17:02:25.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-22T17:02:25.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-22T17:02:25.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-22T17:02:25.808 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-22T17:02:25.809 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-22T17:02:25.809 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-22T17:02:25.809 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-22T17:02:25.809 INFO:teuthology.orchestra.run.smithi047.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-22T17:02:25.809 INFO:teuthology.orchestra.run.smithi047.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-22T17:02:25.809 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-22T17:02:25.809 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-22T17:02:25.810 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-22T17:02:25.810 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-22T17:02:25.810 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-22T17:02:25.810 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.810 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.810 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:02:25.811 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-22T17:02:25.811 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-22T17:02:25.811 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-22T17:02:25.812 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-22T17:02:25.813 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-22T17:02:25.813 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:02:25.813 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2023-06-22T17:02:26.064 DEBUG:teuthology.parallel:result is None 2023-06-22T17:02:27.358 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T17:02:27.601 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T17:02:28.209 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T17:02:28.462 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-22T17:02:28.561 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-22T17:02:28.701 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-22T17:02:46.550 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-22T17:02:46.550 INFO:teuthology.orchestra.run.smithi202.stdout:skipping the directory /sys 2023-06-22T17:02:46.551 INFO:teuthology.orchestra.run.smithi202.stdout:skipping the directory /proc 2023-06-22T17:02:46.551 INFO:teuthology.orchestra.run.smithi202.stdout:skipping the directory /dev 2023-06-22T17:02:46.551 INFO:teuthology.orchestra.run.smithi202.stdout:skipping the directory /run 2023-06-22T17:02:46.552 INFO:teuthology.orchestra.run.smithi202.stdout:skipping the directory /mnt 2023-06-22T17:02:46.552 INFO:teuthology.orchestra.run.smithi202.stdout:skipping the directory /var/tmp 2023-06-22T17:02:46.553 INFO:teuthology.orchestra.run.smithi202.stdout:skipping the directory /home 2023-06-22T17:02:46.553 INFO:teuthology.orchestra.run.smithi202.stdout:skipping the directory /tmp 2023-06-22T17:02:46.553 INFO:teuthology.orchestra.run.smithi202.stdout:skipping the directory /dev 2023-06-22T17:02:46.554 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:02:46.914 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-22T17:02:48.401 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-22T17:02:48.558 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-22T17:02:48.781 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-22T17:02:48.914 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-22T17:02:49.304 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-22T17:02:49.489 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-22T17:02:49.506 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-22T17:02:52.260 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-22T17:02:52.519 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-22T17:02:52.627 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-22T17:02:52.838 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-22T17:02:53.029 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-22T17:02:53.244 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-22T17:02:53.474 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-22T17:02:54.443 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-22T17:02:54.613 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-22T17:02:54.712 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-22T17:02:55.055 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-22T17:02:55.216 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-22T17:02:55.253 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-22T17:02:55.437 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-22T17:02:55.449 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-22T17:02:57.026 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-22T17:02:57.240 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-22T17:02:57.352 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-22T17:02:57.479 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-22T17:02:57.522 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-22T17:02:57.696 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-22T17:02:58.046 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-22T17:02:58.195 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-22T17:03:03.135 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-22T17:03:04.105 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-22T17:03:04.495 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-06-22T17:03:04.708 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 110/122 2023-06-22T17:03:04.968 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-06-22T17:03:05.119 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-06-22T17:03:05.131 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-06-22T17:03:05.324 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-22T17:03:05.466 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-22T17:03:05.661 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-22T17:03:05.837 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-22T17:03:06.103 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-22T17:03:06.319 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-06-22T17:03:06.375 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-06-22T17:03:06.578 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-06-22T17:03:06.587 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-06-22T17:03:06.696 INFO:teuthology.orchestra.run.smithi202.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-06-22T17:03:06.697 INFO:teuthology.orchestra.run.smithi202.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-22T17:03:06.922 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-22T17:03:06.922 INFO:teuthology.orchestra.run.smithi202.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T17:03:07.462 INFO:teuthology.orchestra.run.smithi202.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T17:03:07.462 INFO:teuthology.orchestra.run.smithi202.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-22T17:03:07.462 INFO:teuthology.orchestra.run.smithi202.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-22T17:03:07.462 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:07.773 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-22T17:03:07.773 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-22T17:03:07.774 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-22T17:03:07.774 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T17:03:07.774 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-22T17:03:07.774 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T17:03:07.774 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T17:03:07.774 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-22T17:03:07.774 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-22T17:03:07.775 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T17:03:07.775 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-22T17:03:07.775 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-22T17:03:07.775 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T17:03:07.775 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-22T17:03:07.776 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-22T17:03:07.776 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-22T17:03:07.776 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-22T17:03:07.776 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-22T17:03:07.777 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-22T17:03:07.777 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-22T17:03:07.777 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-22T17:03:07.777 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-22T17:03:07.778 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-22T17:03:07.778 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-22T17:03:07.778 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-22T17:03:07.787 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-22T17:03:07.787 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-22T17:03:07.787 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-22T17:03:07.788 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-22T17:03:07.788 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-22T17:03:07.788 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-22T17:03:07.788 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-22T17:03:07.788 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-22T17:03:07.789 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-22T17:03:07.789 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-22T17:03:07.789 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-22T17:03:07.789 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-22T17:03:07.790 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-22T17:03:07.790 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-22T17:03:07.790 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-22T17:03:07.791 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-22T17:03:07.791 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-22T17:03:07.791 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-22T17:03:07.791 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-22T17:03:07.792 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-22T17:03:07.792 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-22T17:03:07.792 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-22T17:03:07.792 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-22T17:03:07.793 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-22T17:03:07.793 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-22T17:03:07.793 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-22T17:03:07.793 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-22T17:03:07.793 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-22T17:03:07.793 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-22T17:03:07.794 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-22T17:03:07.794 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-22T17:03:07.794 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-22T17:03:07.794 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-22T17:03:07.795 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-22T17:03:07.795 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-22T17:03:07.795 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-22T17:03:07.795 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-22T17:03:07.796 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-22T17:03:07.796 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-22T17:03:07.796 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-22T17:03:07.796 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-22T17:03:07.796 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-22T17:03:07.797 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-22T17:03:07.797 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-22T17:03:07.797 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-22T17:03:07.797 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-22T17:03:07.797 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-06-22T17:03:07.798 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-06-22T17:03:07.798 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-06-22T17:03:07.798 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-06-22T17:03:07.798 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-06-22T17:03:07.798 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-06-22T17:03:07.798 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-06-22T17:03:07.799 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-06-22T17:03:07.799 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-06-22T17:03:07.799 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-06-22T17:03:07.800 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-06-22T17:03:07.800 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-06-22T17:03:07.800 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-06-22T17:03:07.801 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-06-22T17:03:07.801 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-06-22T17:03:07.801 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-06-22T17:03:07.801 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-06-22T17:03:07.802 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-06-22T17:03:07.802 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-06-22T17:03:07.802 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-06-22T17:03:07.803 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-06-22T17:03:07.803 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-06-22T17:03:07.803 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-06-22T17:03:07.803 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-06-22T17:03:07.803 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-06-22T17:03:07.804 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-06-22T17:03:07.804 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-06-22T17:03:07.804 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-06-22T17:03:07.804 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-06-22T17:03:07.805 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-06-22T17:03:07.805 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-06-22T17:03:07.805 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-06-22T17:03:07.806 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-06-22T17:03:07.806 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-06-22T17:03:07.806 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-06-22T17:03:07.806 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-06-22T17:03:07.806 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-06-22T17:03:07.806 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-06-22T17:03:07.807 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-06-22T17:03:07.807 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-06-22T17:03:07.807 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-06-22T17:03:07.807 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-06-22T17:03:07.807 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-06-22T17:03:07.807 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-06-22T17:03:07.808 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-06-22T17:03:07.808 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-06-22T17:03:07.808 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-06-22T17:03:07.808 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-22T17:03:07.808 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-22T17:03:07.809 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-22T17:03:10.002 INFO:teuthology.orchestra.run.smithi202.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T17:03:10.002 INFO:teuthology.orchestra.run.smithi202.stdout:Last metadata expiration check: 0:01:41 ago on Thu 22 Jun 2023 05:01:29 PM UTC. 2023-06-22T17:03:12.176 INFO:teuthology.orchestra.run.smithi202.stdout:Installed products updated. 2023-06-22T17:03:12.323 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:12.323 INFO:teuthology.orchestra.run.smithi202.stdout:Upgraded: 2023-06-22T17:03:12.323 INFO:teuthology.orchestra.run.smithi202.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.324 INFO:teuthology.orchestra.run.smithi202.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.324 INFO:teuthology.orchestra.run.smithi202.stdout:Installed: 2023-06-22T17:03:12.324 INFO:teuthology.orchestra.run.smithi202.stdout: automake-1.16.1-7.el8.noarch 2023-06-22T17:03:12.324 INFO:teuthology.orchestra.run.smithi202.stdout: bison-3.0.4-10.el8.x86_64 2023-06-22T17:03:12.325 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.325 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.325 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.325 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.325 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:03:12.325 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.325 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.326 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.326 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:03:12.326 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:03:12.326 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:03:12.326 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:03:12.326 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:03:12.327 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.327 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.327 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:03:12.327 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.327 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.327 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.327 INFO:teuthology.orchestra.run.smithi202.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:03:12.328 INFO:teuthology.orchestra.run.smithi202.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:03:12.328 INFO:teuthology.orchestra.run.smithi202.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.328 INFO:teuthology.orchestra.run.smithi202.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T17:03:12.328 INFO:teuthology.orchestra.run.smithi202.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-22T17:03:12.328 INFO:teuthology.orchestra.run.smithi202.stdout: flex-2.6.1-9.el8.x86_64 2023-06-22T17:03:12.329 INFO:teuthology.orchestra.run.smithi202.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-22T17:03:12.329 INFO:teuthology.orchestra.run.smithi202.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-22T17:03:12.329 INFO:teuthology.orchestra.run.smithi202.stdout: jq-1.5-12.el8.x86_64 2023-06-22T17:03:12.329 INFO:teuthology.orchestra.run.smithi202.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-22T17:03:12.329 INFO:teuthology.orchestra.run.smithi202.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-22T17:03:12.329 INFO:teuthology.orchestra.run.smithi202.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-22T17:03:12.330 INFO:teuthology.orchestra.run.smithi202.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.330 INFO:teuthology.orchestra.run.smithi202.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.330 INFO:teuthology.orchestra.run.smithi202.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.330 INFO:teuthology.orchestra.run.smithi202.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-22T17:03:12.330 INFO:teuthology.orchestra.run.smithi202.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-22T17:03:12.331 INFO:teuthology.orchestra.run.smithi202.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-22T17:03:12.331 INFO:teuthology.orchestra.run.smithi202.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-22T17:03:12.331 INFO:teuthology.orchestra.run.smithi202.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-22T17:03:12.331 INFO:teuthology.orchestra.run.smithi202.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.331 INFO:teuthology.orchestra.run.smithi202.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.331 INFO:teuthology.orchestra.run.smithi202.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-22T17:03:12.332 INFO:teuthology.orchestra.run.smithi202.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.332 INFO:teuthology.orchestra.run.smithi202.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-22T17:03:12.332 INFO:teuthology.orchestra.run.smithi202.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-22T17:03:12.332 INFO:teuthology.orchestra.run.smithi202.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-22T17:03:12.332 INFO:teuthology.orchestra.run.smithi202.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-22T17:03:12.332 INFO:teuthology.orchestra.run.smithi202.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-22T17:03:12.333 INFO:teuthology.orchestra.run.smithi202.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-22T17:03:12.333 INFO:teuthology.orchestra.run.smithi202.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-22T17:03:12.333 INFO:teuthology.orchestra.run.smithi202.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-22T17:03:12.333 INFO:teuthology.orchestra.run.smithi202.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-22T17:03:12.333 INFO:teuthology.orchestra.run.smithi202.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-22T17:03:12.333 INFO:teuthology.orchestra.run.smithi202.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-22T17:03:12.334 INFO:teuthology.orchestra.run.smithi202.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-22T17:03:12.334 INFO:teuthology.orchestra.run.smithi202.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-22T17:03:12.334 INFO:teuthology.orchestra.run.smithi202.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-22T17:03:12.334 INFO:teuthology.orchestra.run.smithi202.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-22T17:03:12.334 INFO:teuthology.orchestra.run.smithi202.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-22T17:03:12.334 INFO:teuthology.orchestra.run.smithi202.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-22T17:03:12.335 INFO:teuthology.orchestra.run.smithi202.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-22T17:03:12.335 INFO:teuthology.orchestra.run.smithi202.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-22T17:03:12.335 INFO:teuthology.orchestra.run.smithi202.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.335 INFO:teuthology.orchestra.run.smithi202.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.335 INFO:teuthology.orchestra.run.smithi202.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.335 INFO:teuthology.orchestra.run.smithi202.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-22T17:03:12.336 INFO:teuthology.orchestra.run.smithi202.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-22T17:03:12.336 INFO:teuthology.orchestra.run.smithi202.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-22T17:03:12.336 INFO:teuthology.orchestra.run.smithi202.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-22T17:03:12.336 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-22T17:03:12.336 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-22T17:03:12.336 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-22T17:03:12.336 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-22T17:03:12.337 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-22T17:03:12.337 INFO:teuthology.orchestra.run.smithi202.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-22T17:03:12.337 INFO:teuthology.orchestra.run.smithi202.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-22T17:03:12.337 INFO:teuthology.orchestra.run.smithi202.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-22T17:03:12.337 INFO:teuthology.orchestra.run.smithi202.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-22T17:03:12.337 INFO:teuthology.orchestra.run.smithi202.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-22T17:03:12.337 INFO:teuthology.orchestra.run.smithi202.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-22T17:03:12.338 INFO:teuthology.orchestra.run.smithi202.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-22T17:03:12.338 INFO:teuthology.orchestra.run.smithi202.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-22T17:03:12.338 INFO:teuthology.orchestra.run.smithi202.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-22T17:03:12.338 INFO:teuthology.orchestra.run.smithi202.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-22T17:03:12.338 INFO:teuthology.orchestra.run.smithi202.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-22T17:03:12.338 INFO:teuthology.orchestra.run.smithi202.stdout: python3-portend-2.6-1.el8.noarch 2023-06-22T17:03:12.338 INFO:teuthology.orchestra.run.smithi202.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-22T17:03:12.339 INFO:teuthology.orchestra.run.smithi202.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-22T17:03:12.339 INFO:teuthology.orchestra.run.smithi202.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-22T17:03:12.339 INFO:teuthology.orchestra.run.smithi202.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-22T17:03:12.339 INFO:teuthology.orchestra.run.smithi202.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.339 INFO:teuthology.orchestra.run.smithi202.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.339 INFO:teuthology.orchestra.run.smithi202.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-22T17:03:12.339 INFO:teuthology.orchestra.run.smithi202.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-22T17:03:12.340 INFO:teuthology.orchestra.run.smithi202.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.340 INFO:teuthology.orchestra.run.smithi202.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-22T17:03:12.340 INFO:teuthology.orchestra.run.smithi202.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-22T17:03:12.340 INFO:teuthology.orchestra.run.smithi202.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-22T17:03:12.340 INFO:teuthology.orchestra.run.smithi202.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-22T17:03:12.340 INFO:teuthology.orchestra.run.smithi202.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-22T17:03:12.341 INFO:teuthology.orchestra.run.smithi202.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-22T17:03:12.341 INFO:teuthology.orchestra.run.smithi202.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-22T17:03:12.341 INFO:teuthology.orchestra.run.smithi202.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-22T17:03:12.341 INFO:teuthology.orchestra.run.smithi202.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-22T17:03:12.341 INFO:teuthology.orchestra.run.smithi202.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-22T17:03:12.341 INFO:teuthology.orchestra.run.smithi202.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-22T17:03:12.341 INFO:teuthology.orchestra.run.smithi202.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-22T17:03:12.342 INFO:teuthology.orchestra.run.smithi202.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-22T17:03:12.342 INFO:teuthology.orchestra.run.smithi202.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.342 INFO:teuthology.orchestra.run.smithi202.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.342 INFO:teuthology.orchestra.run.smithi202.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T17:03:12.342 INFO:teuthology.orchestra.run.smithi202.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-22T17:03:12.342 INFO:teuthology.orchestra.run.smithi202.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-22T17:03:12.342 INFO:teuthology.orchestra.run.smithi202.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-22T17:03:12.343 INFO:teuthology.orchestra.run.smithi202.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-22T17:03:12.343 INFO:teuthology.orchestra.run.smithi202.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-22T17:03:12.343 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:12.344 INFO:teuthology.orchestra.run.smithi202.stdout:Complete! 2023-06-22T17:03:12.593 DEBUG:teuthology.parallel:result is None 2023-06-22T17:03:12.594 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T17:03:12.818 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-22T17:03:12.845 INFO:teuthology.orchestra.run.smithi047.stdout:17.2.6-719.g42ddc46e.el8 2023-06-22T17:03:12.845 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-22T17:03:12.846 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-22T17:03:12.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T17:03:13.063 DEBUG:teuthology.orchestra.run.smithi202:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-22T17:03:13.084 INFO:teuthology.orchestra.run.smithi202.stdout:17.2.6-719.g42ddc46e.el8 2023-06-22T17:03:13.085 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-22T17:03:13.085 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-22T17:03:13.085 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-22T17:03:13.086 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:13.086 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-22T17:03:13.131 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:13.132 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-22T17:03:13.172 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-22T17:03:13.173 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:13.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2023-06-22T17:03:13.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-22T17:03:13.268 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:13.269 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd of=/usr/bin/daemon-helper 2023-06-22T17:03:13.292 DEBUG:teuthology.orchestra.run.smithi202:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-22T17:03:13.357 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-22T17:03:13.358 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:13.358 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-22T17:03:13.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-22T17:03:13.452 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:13.453 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-22T17:03:13.477 DEBUG:teuthology.orchestra.run.smithi202:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-22T17:03:13.541 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-22T17:03:13.541 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:13.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2023-06-22T17:03:13.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-22T17:03:13.636 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:13.637 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd of=/usr/bin/stdin-killer 2023-06-22T17:03:13.661 DEBUG:teuthology.orchestra.run.smithi202:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-22T17:03:13.724 INFO:teuthology.run_tasks:Running task ceph... 2023-06-22T17:03:13.803 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-06-22T17:03:13.804 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2023-06-22T17:03:13.806 DEBUG:teuthology.orchestra.run.smithi202:> sudo chmod 777 /var/log/ceph 2023-06-22T17:03:13.830 INFO:tasks.ceph:Disabling ceph logrotate... 2023-06-22T17:03:13.831 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-22T17:03:13.873 DEBUG:teuthology.orchestra.run.smithi202:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-22T17:03:13.897 INFO:tasks.ceph:Creating extra log directories... 2023-06-22T17:03:13.897 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-22T17:03:13.940 DEBUG:teuthology.orchestra.run.smithi202:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-22T17:03:13.975 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-06-22T17:03:13.975 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-06-22T17:03:13.976 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-06-22T17:03:13.976 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:13.976 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-22T17:03:14.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-22T17:03:14.134 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-22T17:03:14.161 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:14.161 DEBUG:teuthology.orchestra.run.smithi202:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-22T17:03:14.176 DEBUG:teuthology.orchestra.run.smithi202:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-22T17:03:14.289 DEBUG:teuthology.orchestra.run.smithi202:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-22T17:03:14.319 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-06-22T17:03:14.320 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-06-22T17:03:14.320 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814', 'branch': 'wip-yuri4-testing-2023-06-21-0846-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7311814', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2023-06-21-0846-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri4-testing-2023-06-21-0846-quincy', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9037, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri4-testing-2023-06-21-0846-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'targets': {'smithi047.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlSCLbZxCowylv4ejcGPoLZwSzztkWsoXph8jwxtsLJ3VXlph3F9wrR1ghn5qjjUtVXhuKxEgDbyX2uINaPFag=', 'smithi202.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDypvRW1ScAmzbCqMdu2b44iXhoAYmmsMtSCXCqMi+j9mOvi4XOcXfKNu/BGvrd49WtXo1PRU4OHW4zMvGMex9eKutRgJWxD9Y3HnFOXkfIaKeFe6odwFQ1RiPLLCSWoeZ5GeIqpqEs9xMEusoo4ly25clw6Ex+7chqlCXWcqt3aVasVUB+46NzjXHQtiPfZn9euC07pub6yE5fowjee3DrA/f8wRRhcXr6EZPuupJkEwQlgwz+dtEA9TG6x/WbPTqyYZ6O4B0Qv9/7IavdhpBLFwH/RbKwnMHDc4drAWjZf+WX65YOMNDvCejLaXT2eyStMp8JisTVu8vXlvUZ7fQjJ29WD3pvzjxHKh5mQdQbYV0m3L2EEBZvLDEidx0YJmlXkZcanoRDbSEeEMHuUP/5nc2OU7OmdEleiydSK1jfVkVyYAbfz7RNpajUaBehbvyxnB5d1Oy1E41DoIEnjIks5Ee4tj6Im2iqIA4bhjyMi8rCjelVj5/8xoJb2AteqYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots', 'tasks.cephfs.test_volumes.TestSubvolumeSnapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '076bbebc42a14f7d568aaa78eabb0038327bcb23', 'timestamp': '2023-06-22_15:25:25', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-06-22T17:03:14.321 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-22T17:03:14.323 DEBUG:teuthology.orchestra.run.smithi202:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-22T17:03:14.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2023-06-22T17:03:14.379 DEBUG:teuthology.orchestra.run.smithi202:> sudo install -d -m0777 -- /var/run/ceph 2023-06-22T17:03:14.432 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:14.432 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2023-06-22T17:03:14.456 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-22T17:03:14.457 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2023-06-22T17:03:14.512 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-22T17:03:14.513 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T17:03:14.513 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 208524 Links: 1 2023-06-22T17:03:14.513 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T17:03:14.514 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T17:03:14.514 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2023-06-22 17:02:20.675893313 +0000 2023-06-22T17:03:14.514 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2023-06-22 16:59:43.818873807 +0000 2023-06-22T17:03:14.515 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2023-06-22 16:59:43.818873807 +0000 2023-06-22T17:03:14.515 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2023-06-22T17:03:14.516 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-22T17:03:14.578 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2023-06-22T17:03:14.578 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2023-06-22T17:03:14.579 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000118023 s, 4.3 MB/s 2023-06-22T17:03:14.579 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-22T17:03:14.634 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2023-06-22T17:03:14.689 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-22T17:03:14.690 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T17:03:14.690 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 213415 Links: 1 2023-06-22T17:03:14.690 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T17:03:14.690 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T17:03:14.690 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2023-06-22 17:02:20.675893313 +0000 2023-06-22T17:03:14.690 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2023-06-22 16:59:44.309863546 +0000 2023-06-22T17:03:14.691 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2023-06-22 16:59:44.309863546 +0000 2023-06-22T17:03:14.691 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2023-06-22T17:03:14.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-22T17:03:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2023-06-22T17:03:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2023-06-22T17:03:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000105833 s, 4.8 MB/s 2023-06-22T17:03:14.754 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-22T17:03:14.810 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2023-06-22T17:03:14.866 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-22T17:03:14.866 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T17:03:14.866 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 210892 Links: 1 2023-06-22T17:03:14.867 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T17:03:14.867 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T17:03:14.867 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2023-06-22 17:02:20.675893313 +0000 2023-06-22T17:03:14.867 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2023-06-22 16:59:44.811853054 +0000 2023-06-22T17:03:14.867 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2023-06-22 16:59:44.811853054 +0000 2023-06-22T17:03:14.867 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2023-06-22T17:03:14.868 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-22T17:03:14.929 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2023-06-22T17:03:14.929 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2023-06-22T17:03:14.929 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000110918 s, 4.6 MB/s 2023-06-22T17:03:14.930 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-22T17:03:14.986 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2023-06-22T17:03:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-22T17:03:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T17:03:15.043 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 215076 Links: 1 2023-06-22T17:03:15.043 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T17:03:15.043 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T17:03:15.044 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2023-06-22 17:02:20.675893313 +0000 2023-06-22T17:03:15.044 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2023-06-22 16:59:45.320842417 +0000 2023-06-22T17:03:15.045 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2023-06-22 16:59:45.320842417 +0000 2023-06-22T17:03:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2023-06-22T17:03:15.046 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-22T17:03:15.105 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2023-06-22T17:03:15.106 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2023-06-22T17:03:15.106 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.9298e-05 s, 5.2 MB/s 2023-06-22T17:03:15.107 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-22T17:03:15.163 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-22T17:03:15.163 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:15.164 DEBUG:teuthology.orchestra.run.smithi202:> dd if=/scratch_devs of=/dev/stdout 2023-06-22T17:03:15.181 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-22T17:03:15.181 DEBUG:teuthology.orchestra.run.smithi202:> stat /dev/vg_nvme/lv_1 2023-06-22T17:03:15.236 INFO:teuthology.orchestra.run.smithi202.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-22T17:03:15.236 INFO:teuthology.orchestra.run.smithi202.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T17:03:15.236 INFO:teuthology.orchestra.run.smithi202.stdout:Device: 6h/6d Inode: 212494 Links: 1 2023-06-22T17:03:15.236 INFO:teuthology.orchestra.run.smithi202.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T17:03:15.237 INFO:teuthology.orchestra.run.smithi202.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T17:03:15.237 INFO:teuthology.orchestra.run.smithi202.stdout:Access: 2023-06-22 17:03:07.377573378 +0000 2023-06-22T17:03:15.237 INFO:teuthology.orchestra.run.smithi202.stdout:Modify: 2023-06-22 16:59:33.210971288 +0000 2023-06-22T17:03:15.237 INFO:teuthology.orchestra.run.smithi202.stdout:Change: 2023-06-22 16:59:33.210971288 +0000 2023-06-22T17:03:15.237 INFO:teuthology.orchestra.run.smithi202.stdout: Birth: - 2023-06-22T17:03:15.237 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-22T17:03:15.299 INFO:teuthology.orchestra.run.smithi202.stderr:1+0 records in 2023-06-22T17:03:15.299 INFO:teuthology.orchestra.run.smithi202.stderr:1+0 records out 2023-06-22T17:03:15.299 INFO:teuthology.orchestra.run.smithi202.stderr:512 bytes copied, 0.000280757 s, 1.8 MB/s 2023-06-22T17:03:15.300 DEBUG:teuthology.orchestra.run.smithi202:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-22T17:03:15.356 DEBUG:teuthology.orchestra.run.smithi202:> stat /dev/vg_nvme/lv_2 2023-06-22T17:03:15.410 INFO:teuthology.orchestra.run.smithi202.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-22T17:03:15.411 INFO:teuthology.orchestra.run.smithi202.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T17:03:15.411 INFO:teuthology.orchestra.run.smithi202.stdout:Device: 6h/6d Inode: 210719 Links: 1 2023-06-22T17:03:15.411 INFO:teuthology.orchestra.run.smithi202.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T17:03:15.411 INFO:teuthology.orchestra.run.smithi202.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T17:03:15.411 INFO:teuthology.orchestra.run.smithi202.stdout:Access: 2023-06-22 17:03:07.377573378 +0000 2023-06-22T17:03:15.412 INFO:teuthology.orchestra.run.smithi202.stdout:Modify: 2023-06-22 16:59:33.701962318 +0000 2023-06-22T17:03:15.412 INFO:teuthology.orchestra.run.smithi202.stdout:Change: 2023-06-22 16:59:33.701962318 +0000 2023-06-22T17:03:15.412 INFO:teuthology.orchestra.run.smithi202.stdout: Birth: - 2023-06-22T17:03:15.412 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-22T17:03:15.474 INFO:teuthology.orchestra.run.smithi202.stderr:1+0 records in 2023-06-22T17:03:15.474 INFO:teuthology.orchestra.run.smithi202.stderr:1+0 records out 2023-06-22T17:03:15.474 INFO:teuthology.orchestra.run.smithi202.stderr:512 bytes copied, 0.000246883 s, 2.1 MB/s 2023-06-22T17:03:15.475 DEBUG:teuthology.orchestra.run.smithi202:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-22T17:03:15.530 DEBUG:teuthology.orchestra.run.smithi202:> stat /dev/vg_nvme/lv_3 2023-06-22T17:03:15.584 INFO:teuthology.orchestra.run.smithi202.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-22T17:03:15.585 INFO:teuthology.orchestra.run.smithi202.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T17:03:15.585 INFO:teuthology.orchestra.run.smithi202.stdout:Device: 6h/6d Inode: 205437 Links: 1 2023-06-22T17:03:15.585 INFO:teuthology.orchestra.run.smithi202.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T17:03:15.585 INFO:teuthology.orchestra.run.smithi202.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T17:03:15.585 INFO:teuthology.orchestra.run.smithi202.stdout:Access: 2023-06-22 17:03:07.377573378 +0000 2023-06-22T17:03:15.586 INFO:teuthology.orchestra.run.smithi202.stdout:Modify: 2023-06-22 16:59:34.202953164 +0000 2023-06-22T17:03:15.586 INFO:teuthology.orchestra.run.smithi202.stdout:Change: 2023-06-22 16:59:34.202953164 +0000 2023-06-22T17:03:15.586 INFO:teuthology.orchestra.run.smithi202.stdout: Birth: - 2023-06-22T17:03:15.586 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-22T17:03:15.648 INFO:teuthology.orchestra.run.smithi202.stderr:1+0 records in 2023-06-22T17:03:15.648 INFO:teuthology.orchestra.run.smithi202.stderr:1+0 records out 2023-06-22T17:03:15.648 INFO:teuthology.orchestra.run.smithi202.stderr:512 bytes copied, 0.000219058 s, 2.3 MB/s 2023-06-22T17:03:15.649 DEBUG:teuthology.orchestra.run.smithi202:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-22T17:03:15.705 DEBUG:teuthology.orchestra.run.smithi202:> stat /dev/vg_nvme/lv_4 2023-06-22T17:03:15.760 INFO:teuthology.orchestra.run.smithi202.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-22T17:03:15.760 INFO:teuthology.orchestra.run.smithi202.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T17:03:15.761 INFO:teuthology.orchestra.run.smithi202.stdout:Device: 6h/6d Inode: 212605 Links: 1 2023-06-22T17:03:15.761 INFO:teuthology.orchestra.run.smithi202.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T17:03:15.761 INFO:teuthology.orchestra.run.smithi202.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T17:03:15.761 INFO:teuthology.orchestra.run.smithi202.stdout:Access: 2023-06-22 17:03:07.378573360 +0000 2023-06-22T17:03:15.762 INFO:teuthology.orchestra.run.smithi202.stdout:Modify: 2023-06-22 16:59:34.878940813 +0000 2023-06-22T17:03:15.762 INFO:teuthology.orchestra.run.smithi202.stdout:Change: 2023-06-22 16:59:34.878940813 +0000 2023-06-22T17:03:15.762 INFO:teuthology.orchestra.run.smithi202.stdout: Birth: - 2023-06-22T17:03:15.763 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-22T17:03:15.822 INFO:teuthology.orchestra.run.smithi202.stderr:1+0 records in 2023-06-22T17:03:15.823 INFO:teuthology.orchestra.run.smithi202.stderr:1+0 records out 2023-06-22T17:03:15.823 INFO:teuthology.orchestra.run.smithi202.stderr:512 bytes copied, 0.000219911 s, 2.3 MB/s 2023-06-22T17:03:15.824 DEBUG:teuthology.orchestra.run.smithi202:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-22T17:03:15.879 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-22T17:03:15.879 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi202.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-22T17:03:15.880 INFO:tasks.ceph:Generating config... 2023-06-22T17:03:15.881 INFO:tasks.ceph:[client] client mount timeout = 600 2023-06-22T17:03:15.881 INFO:tasks.ceph:[client] debug client = 20 2023-06-22T17:03:15.881 INFO:tasks.ceph:[client] debug ms = 1 2023-06-22T17:03:15.882 INFO:tasks.ceph:[client] pid file = 2023-06-22T17:03:15.882 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-06-22T17:03:15.882 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-06-22T17:03:15.882 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-06-22T17:03:15.882 INFO:tasks.ceph:[mds] debug mds = 20 2023-06-22T17:03:15.883 INFO:tasks.ceph:[mds] debug ms = 1 2023-06-22T17:03:15.883 INFO:tasks.ceph:[mds] mds debug frag = True 2023-06-22T17:03:15.883 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-06-22T17:03:15.883 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-06-22T17:03:15.883 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-06-22T17:03:15.884 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-06-22T17:03:15.884 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-06-22T17:03:15.884 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-06-22T17:03:15.884 INFO:tasks.ceph:[mgr] debug client = 20 2023-06-22T17:03:15.884 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-06-22T17:03:15.885 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-06-22T17:03:15.885 INFO:tasks.ceph:[mgr] debug ms = 1 2023-06-22T17:03:15.885 INFO:tasks.ceph:[mon] debug mon = 20 2023-06-22T17:03:15.885 INFO:tasks.ceph:[mon] debug ms = 1 2023-06-22T17:03:15.885 INFO:tasks.ceph:[mon] debug paxos = 20 2023-06-22T17:03:15.886 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-06-22T17:03:15.886 INFO:tasks.ceph:[osd] bdev async discard = True 2023-06-22T17:03:15.886 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-06-22T17:03:15.886 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-06-22T17:03:15.886 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-06-22T17:03:15.886 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-06-22T17:03:15.887 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-06-22T17:03:15.887 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-06-22T17:03:15.887 INFO:tasks.ceph:[osd] debug ms = 1 2023-06-22T17:03:15.887 INFO:tasks.ceph:[osd] debug osd = 20 2023-06-22T17:03:15.887 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-06-22T17:03:15.888 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-06-22T17:03:15.888 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-06-22T17:03:15.888 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-06-22T17:03:15.888 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-06-22T17:03:15.888 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-06-22T17:03:15.889 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-06-22T17:03:15.889 INFO:tasks.ceph:Setting up mon.a... 2023-06-22T17:03:15.889 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-06-22T17:03:15.939 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.keyring 2023-06-22T17:03:15.942 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-06-22T17:03:15.983 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-22T17:03:16.049 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.47'), ('mon.b', '172.21.15.202'), ('mon.c', '[v2:172.21.15.202:3301,v1:172.21.15.202:6790]')] 2023-06-22T17:03:16.050 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.47,172.21.15.202,[v2:172.21.15.202:3301,v1:172.21.15.202:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-06-22T17:03:16.051 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:16.051 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-06-22T17:03:16.104 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.47 --add b 172.21.15.202 --addv c '[v2:172.21.15.202:3301,v1:172.21.15.202:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:16.188 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:16.188 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: generated fsid 61e931aa-8dfe-4361-8a29-1d30432b403b 2023-06-22T17:03:16.188 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = octopus 2023-06-22T17:03:16.189 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 0 2023-06-22T17:03:16.189 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 61e931aa-8dfe-4361-8a29-1d30432b403b 2023-06-22T17:03:16.189 INFO:teuthology.orchestra.run.smithi047.stdout:last_changed 2023-06-22T17:03:16.187779+0000 2023-06-22T17:03:16.190 INFO:teuthology.orchestra.run.smithi047.stdout:created 2023-06-22T17:03:16.187779+0000 2023-06-22T17:03:16.190 INFO:teuthology.orchestra.run.smithi047.stdout:min_mon_release 15 (octopus) 2023-06-22T17:03:16.190 INFO:teuthology.orchestra.run.smithi047.stdout:election_strategy: 1 2023-06-22T17:03:16.190 INFO:teuthology.orchestra.run.smithi047.stdout:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2023-06-22T17:03:16.191 INFO:teuthology.orchestra.run.smithi047.stdout:1: [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] mon.b 2023-06-22T17:03:16.191 INFO:teuthology.orchestra.run.smithi047.stdout:2: [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] mon.c 2023-06-22T17:03:16.192 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-06-22T17:03:16.193 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-06-22T17:03:16.248 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 61e931aa-8dfe-4361-8a29-1d30432b403b... 2023-06-22T17:03:16.250 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-22T17:03:16.291 DEBUG:teuthology.orchestra.run.smithi202:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-22T17:03:16.333 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2023-06-22T17:03:16.333 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2023-06-22T17:03:16.333 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-22T17:03:16.333 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2023-06-22T17:03:16.333 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.333 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2023-06-22T17:03:16.334 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.334 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2023-06-22T17:03:16.334 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.334 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2023-06-22T17:03:16.334 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2023-06-22T17:03:16.334 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.335 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2023-06-22T17:03:16.335 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2023-06-22T17:03:16.335 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.335 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-22T17:03:16.335 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2023-06-22T17:03:16.335 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.335 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2023-06-22T17:03:16.336 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-22T17:03:16.336 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.336 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2023-06-22T17:03:16.336 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.336 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2023-06-22T17:03:16.336 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2023-06-22T17:03:16.336 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2023-06-22T17:03:16.337 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2023-06-22T17:03:16.337 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2023-06-22T17:03:16.337 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2023-06-22T17:03:16.337 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2023-06-22T17:03:16.337 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-22T17:03:16.337 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-22T17:03:16.337 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2023-06-22T17:03:16.337 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.338 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-22T17:03:16.338 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.338 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2023-06-22T17:03:16.338 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.338 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2023-06-22T17:03:16.338 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.338 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2023-06-22T17:03:16.338 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2023-06-22T17:03:16.339 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2023-06-22T17:03:16.339 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.202,[v2:172.21.15.202:3301,v1:172.21.15.202:6790]" 2023-06-22T17:03:16.339 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on skipped message = False 2023-06-22T17:03:16.339 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 61e931aa-8dfe-4361-8a29-1d30432b403b 2023-06-22T17:03:16.339 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.339 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2023-06-22T17:03:16.339 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2023-06-22T17:03:16.339 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.340 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2023-06-22T17:03:16.340 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2023-06-22T17:03:16.340 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2023-06-22T17:03:16.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.340 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2023-06-22T17:03:16.341 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2023-06-22T17:03:16.341 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.341 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2023-06-22T17:03:16.341 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2023-06-22T17:03:16.341 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2023-06-22T17:03:16.341 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.341 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2023-06-22T17:03:16.341 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2023-06-22T17:03:16.342 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.342 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2023-06-22T17:03:16.342 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.342 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2023-06-22T17:03:16.342 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.343 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2023-06-22T17:03:16.343 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.343 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2023-06-22T17:03:16.343 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2023-06-22T17:03:16.344 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.344 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2023-06-22T17:03:16.344 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2023-06-22T17:03:16.344 INFO:teuthology.orchestra.run.smithi047.stdout: bdev async discard = True 2023-06-22T17:03:16.344 INFO:teuthology.orchestra.run.smithi047.stdout: bdev enable discard = True 2023-06-22T17:03:16.344 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = bitmap 2023-06-22T17:03:16.345 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2023-06-22T17:03:16.345 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2023-06-22T17:03:16.345 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 1/20 2023-06-22T17:03:16.345 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 1/20 2023-06-22T17:03:16.345 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2023-06-22T17:03:16.345 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2023-06-22T17:03:16.345 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 4/10 2023-06-22T17:03:16.345 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2023-06-22T17:03:16.346 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2023-06-22T17:03:16.346 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2023-06-22T17:03:16.346 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2023-06-22T17:03:16.346 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2023-06-22T17:03:16.346 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2023-06-22T17:03:16.346 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.346 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2023-06-22T17:03:16.346 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2023-06-22T17:03:16.347 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2023-06-22T17:03:16.347 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2023-06-22T17:03:16.347 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2023-06-22T17:03:16.347 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2023-06-22T17:03:16.347 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2023-06-22T17:03:16.347 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2023-06-22T17:03:16.347 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2023-06-22T17:03:16.347 INFO:teuthology.orchestra.run.smithi047.stdout: debug finisher = 20 2023-06-22T17:03:16.348 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.348 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2023-06-22T17:03:16.348 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2023-06-22T17:03:16.348 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2023-06-22T17:03:16.348 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2023-06-22T17:03:16.348 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2023-06-22T17:03:16.348 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2023-06-22T17:03:16.349 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2023-06-22T17:03:16.349 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2023-06-22T17:03:16.349 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2023-06-22T17:03:16.349 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2023-06-22T17:03:16.349 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2023-06-22T17:03:16.349 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.350 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-22T17:03:16.350 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2023-06-22T17:03:16.350 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2023-06-22T17:03:16.350 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.350 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2023-06-22T17:03:16.350 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-22T17:03:16.350 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-22T17:03:16.350 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.351 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2023-06-22T17:03:16.351 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2023-06-22T17:03:16.351 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.351 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2023-06-22T17:03:16.351 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2023-06-22T17:03:16.351 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:16.351 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2023-06-22T17:03:16.351 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2023-06-22T17:03:16.352 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2023-06-22T17:03:16.352 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2023-06-22T17:03:16.352 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-22T17:03:16.352 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-22T17:03:16.352 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2023-06-22T17:03:16.352 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2023-06-22T17:03:16.352 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2023-06-22T17:03:16.353 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = "" 2023-06-22T17:03:16.353 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2023-06-22T17:03:16.353 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2023-06-22T17:03:16.353 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2023-06-22T17:03:16.353 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2023-06-22T17:03:16.353 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2023-06-22T17:03:16.354 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2023-06-22T17:03:16.354 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2023-06-22T17:03:16.354 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2023-06-22T17:03:16.354 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2023-06-22T17:03:16.354 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2023-06-22T17:03:16.354 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2023-06-22T17:03:16.355 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2023-06-22T17:03:16.355 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2023-06-22T17:03:16.355 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2023-06-22T17:03:16.355 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2023-06-22T17:03:16.355 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2023-06-22T17:03:16.355 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2023-06-22T17:03:16.356 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2023-06-22T17:03:16.356 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2023-06-22T17:03:16.358 INFO:teuthology.orchestra.run.smithi202.stdout:[global] 2023-06-22T17:03:16.358 INFO:teuthology.orchestra.run.smithi202.stdout: chdir = "" 2023-06-22T17:03:16.359 INFO:teuthology.orchestra.run.smithi202.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-22T17:03:16.359 INFO:teuthology.orchestra.run.smithi202.stdout: auth supported = cephx 2023-06-22T17:03:16.359 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.360 INFO:teuthology.orchestra.run.smithi202.stdout: filestore xattr use omap = true 2023-06-22T17:03:16.360 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.360 INFO:teuthology.orchestra.run.smithi202.stdout: mon clock drift allowed = 1.000 2023-06-22T17:03:16.360 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.361 INFO:teuthology.orchestra.run.smithi202.stdout: osd crush chooseleaf type = 0 2023-06-22T17:03:16.361 INFO:teuthology.orchestra.run.smithi202.stdout: auth debug = true 2023-06-22T17:03:16.361 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.361 INFO:teuthology.orchestra.run.smithi202.stdout: ms die on old message = true 2023-06-22T17:03:16.362 INFO:teuthology.orchestra.run.smithi202.stdout: ms die on bug = true 2023-06-22T17:03:16.362 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.362 INFO:teuthology.orchestra.run.smithi202.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-22T17:03:16.363 INFO:teuthology.orchestra.run.smithi202.stdout: mon pg warn max object skew = 0 2023-06-22T17:03:16.363 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.363 INFO:teuthology.orchestra.run.smithi202.stdout: # disable pg_autoscaler by default for new pools 2023-06-22T17:03:16.363 INFO:teuthology.orchestra.run.smithi202.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-22T17:03:16.363 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.364 INFO:teuthology.orchestra.run.smithi202.stdout: osd pool default size = 2 2023-06-22T17:03:16.364 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.364 INFO:teuthology.orchestra.run.smithi202.stdout: mon osd allow primary affinity = true 2023-06-22T17:03:16.364 INFO:teuthology.orchestra.run.smithi202.stdout: mon osd allow pg remap = true 2023-06-22T17:03:16.365 INFO:teuthology.orchestra.run.smithi202.stdout: mon warn on legacy crush tunables = false 2023-06-22T17:03:16.365 INFO:teuthology.orchestra.run.smithi202.stdout: mon warn on crush straw calc version zero = false 2023-06-22T17:03:16.365 INFO:teuthology.orchestra.run.smithi202.stdout: mon warn on no sortbitwise = false 2023-06-22T17:03:16.365 INFO:teuthology.orchestra.run.smithi202.stdout: mon warn on osd down out interval zero = false 2023-06-22T17:03:16.365 INFO:teuthology.orchestra.run.smithi202.stdout: mon warn on too few osds = false 2023-06-22T17:03:16.365 INFO:teuthology.orchestra.run.smithi202.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-22T17:03:16.366 INFO:teuthology.orchestra.run.smithi202.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-22T17:03:16.366 INFO:teuthology.orchestra.run.smithi202.stdout: mon_allow_pool_size_one = true 2023-06-22T17:03:16.366 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.366 INFO:teuthology.orchestra.run.smithi202.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-22T17:03:16.366 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.366 INFO:teuthology.orchestra.run.smithi202.stdout: osd default data pool replay window = 5 2023-06-22T17:03:16.366 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.367 INFO:teuthology.orchestra.run.smithi202.stdout: mon allow pool delete = true 2023-06-22T17:03:16.367 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.367 INFO:teuthology.orchestra.run.smithi202.stdout: mon cluster log file level = debug 2023-06-22T17:03:16.368 INFO:teuthology.orchestra.run.smithi202.stdout: debug asserts on shutdown = true 2023-06-22T17:03:16.368 INFO:teuthology.orchestra.run.smithi202.stdout: mon health detail to clog = false 2023-06-22T17:03:16.368 INFO:teuthology.orchestra.run.smithi202.stdout: mon host = "172.21.15.47,172.21.15.202,[v2:172.21.15.202:3301,v1:172.21.15.202:6790]" 2023-06-22T17:03:16.368 INFO:teuthology.orchestra.run.smithi202.stdout: ms die on skipped message = False 2023-06-22T17:03:16.369 INFO:teuthology.orchestra.run.smithi202.stdout: fsid = 61e931aa-8dfe-4361-8a29-1d30432b403b 2023-06-22T17:03:16.369 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.369 INFO:teuthology.orchestra.run.smithi202.stdout:[osd] 2023-06-22T17:03:16.369 INFO:teuthology.orchestra.run.smithi202.stdout: osd journal size = 100 2023-06-22T17:03:16.370 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.370 INFO:teuthology.orchestra.run.smithi202.stdout: osd scrub load threshold = 5.0 2023-06-22T17:03:16.370 INFO:teuthology.orchestra.run.smithi202.stdout: osd scrub max interval = 600 2023-06-22T17:03:16.370 INFO:teuthology.orchestra.run.smithi202.stdout: osd mclock profile = high_recovery_ops 2023-06-22T17:03:16.371 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.371 INFO:teuthology.orchestra.run.smithi202.stdout: osd recover clone overlap = true 2023-06-22T17:03:16.371 INFO:teuthology.orchestra.run.smithi202.stdout: osd recovery max chunk = 1048576 2023-06-22T17:03:16.372 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.372 INFO:teuthology.orchestra.run.smithi202.stdout: osd debug shutdown = true 2023-06-22T17:03:16.372 INFO:teuthology.orchestra.run.smithi202.stdout: osd debug op order = true 2023-06-22T17:03:16.372 INFO:teuthology.orchestra.run.smithi202.stdout: osd debug verify stray on activate = true 2023-06-22T17:03:16.373 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.373 INFO:teuthology.orchestra.run.smithi202.stdout: osd open classes on start = true 2023-06-22T17:03:16.373 INFO:teuthology.orchestra.run.smithi202.stdout: osd debug pg log writeout = true 2023-06-22T17:03:16.373 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.374 INFO:teuthology.orchestra.run.smithi202.stdout: osd deep scrub update digest min age = 30 2023-06-22T17:03:16.374 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.374 INFO:teuthology.orchestra.run.smithi202.stdout: osd map max advance = 10 2023-06-22T17:03:16.374 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.375 INFO:teuthology.orchestra.run.smithi202.stdout: journal zero on create = true 2023-06-22T17:03:16.375 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.375 INFO:teuthology.orchestra.run.smithi202.stdout: filestore ondisk finisher threads = 3 2023-06-22T17:03:16.375 INFO:teuthology.orchestra.run.smithi202.stdout: filestore apply finisher threads = 3 2023-06-22T17:03:16.376 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.376 INFO:teuthology.orchestra.run.smithi202.stdout: bdev debug aio = true 2023-06-22T17:03:16.376 INFO:teuthology.orchestra.run.smithi202.stdout: osd debug misdirected ops = true 2023-06-22T17:03:16.376 INFO:teuthology.orchestra.run.smithi202.stdout: bdev async discard = True 2023-06-22T17:03:16.376 INFO:teuthology.orchestra.run.smithi202.stdout: bdev enable discard = True 2023-06-22T17:03:16.376 INFO:teuthology.orchestra.run.smithi202.stdout: bluestore allocator = bitmap 2023-06-22T17:03:16.376 INFO:teuthology.orchestra.run.smithi202.stdout: bluestore block size = 96636764160 2023-06-22T17:03:16.377 INFO:teuthology.orchestra.run.smithi202.stdout: bluestore fsck on mount = True 2023-06-22T17:03:16.377 INFO:teuthology.orchestra.run.smithi202.stdout: debug bluefs = 1/20 2023-06-22T17:03:16.377 INFO:teuthology.orchestra.run.smithi202.stdout: debug bluestore = 1/20 2023-06-22T17:03:16.377 INFO:teuthology.orchestra.run.smithi202.stdout: debug ms = 1 2023-06-22T17:03:16.377 INFO:teuthology.orchestra.run.smithi202.stdout: debug osd = 20 2023-06-22T17:03:16.378 INFO:teuthology.orchestra.run.smithi202.stdout: debug rocksdb = 4/10 2023-06-22T17:03:16.378 INFO:teuthology.orchestra.run.smithi202.stdout: mon osd backfillfull_ratio = 0.85 2023-06-22T17:03:16.378 INFO:teuthology.orchestra.run.smithi202.stdout: mon osd full ratio = 0.9 2023-06-22T17:03:16.378 INFO:teuthology.orchestra.run.smithi202.stdout: mon osd nearfull ratio = 0.8 2023-06-22T17:03:16.379 INFO:teuthology.orchestra.run.smithi202.stdout: osd failsafe full ratio = 0.95 2023-06-22T17:03:16.379 INFO:teuthology.orchestra.run.smithi202.stdout: osd objectstore = bluestore 2023-06-22T17:03:16.379 INFO:teuthology.orchestra.run.smithi202.stdout: osd op complaint time = 180 2023-06-22T17:03:16.380 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.380 INFO:teuthology.orchestra.run.smithi202.stdout:[mgr] 2023-06-22T17:03:16.380 INFO:teuthology.orchestra.run.smithi202.stdout: debug ms = 1 2023-06-22T17:03:16.380 INFO:teuthology.orchestra.run.smithi202.stdout: debug mgr = 20 2023-06-22T17:03:16.381 INFO:teuthology.orchestra.run.smithi202.stdout: debug mon = 20 2023-06-22T17:03:16.381 INFO:teuthology.orchestra.run.smithi202.stdout: debug auth = 20 2023-06-22T17:03:16.381 INFO:teuthology.orchestra.run.smithi202.stdout: mon reweight min pgs per osd = 4 2023-06-22T17:03:16.382 INFO:teuthology.orchestra.run.smithi202.stdout: mon reweight min bytes per osd = 10 2023-06-22T17:03:16.382 INFO:teuthology.orchestra.run.smithi202.stdout: mgr/telemetry/nag = false 2023-06-22T17:03:16.382 INFO:teuthology.orchestra.run.smithi202.stdout: debug client = 20 2023-06-22T17:03:16.383 INFO:teuthology.orchestra.run.smithi202.stdout: debug finisher = 20 2023-06-22T17:03:16.383 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.383 INFO:teuthology.orchestra.run.smithi202.stdout:[mon] 2023-06-22T17:03:16.383 INFO:teuthology.orchestra.run.smithi202.stdout: debug ms = 1 2023-06-22T17:03:16.384 INFO:teuthology.orchestra.run.smithi202.stdout: debug mon = 20 2023-06-22T17:03:16.384 INFO:teuthology.orchestra.run.smithi202.stdout: debug paxos = 20 2023-06-22T17:03:16.384 INFO:teuthology.orchestra.run.smithi202.stdout: debug auth = 20 2023-06-22T17:03:16.385 INFO:teuthology.orchestra.run.smithi202.stdout: mon data avail warn = 5 2023-06-22T17:03:16.385 INFO:teuthology.orchestra.run.smithi202.stdout: mon mgr mkfs grace = 240 2023-06-22T17:03:16.385 INFO:teuthology.orchestra.run.smithi202.stdout: mon reweight min pgs per osd = 4 2023-06-22T17:03:16.385 INFO:teuthology.orchestra.run.smithi202.stdout: mon osd reporter subtree level = osd 2023-06-22T17:03:16.386 INFO:teuthology.orchestra.run.smithi202.stdout: mon osd prime pg temp = true 2023-06-22T17:03:16.386 INFO:teuthology.orchestra.run.smithi202.stdout: mon reweight min bytes per osd = 10 2023-06-22T17:03:16.386 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.386 INFO:teuthology.orchestra.run.smithi202.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-22T17:03:16.386 INFO:teuthology.orchestra.run.smithi202.stdout: auth mon ticket ttl = 660 # 11m 2023-06-22T17:03:16.387 INFO:teuthology.orchestra.run.smithi202.stdout: auth service ticket ttl = 240 # 4m 2023-06-22T17:03:16.387 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.387 INFO:teuthology.orchestra.run.smithi202.stdout: # don't complain about insecure global_id in the test suite 2023-06-22T17:03:16.387 INFO:teuthology.orchestra.run.smithi202.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-22T17:03:16.388 INFO:teuthology.orchestra.run.smithi202.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-22T17:03:16.388 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.388 INFO:teuthology.orchestra.run.smithi202.stdout: # 1m isn't quite enough 2023-06-22T17:03:16.388 INFO:teuthology.orchestra.run.smithi202.stdout: mon_down_mkfs_grace = 2m 2023-06-22T17:03:16.389 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.389 INFO:teuthology.orchestra.run.smithi202.stdout: mon_warn_on_filestore_osds = false 2023-06-22T17:03:16.390 INFO:teuthology.orchestra.run.smithi202.stdout: mon op complaint time = 120 2023-06-22T17:03:16.390 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:03:16.390 INFO:teuthology.orchestra.run.smithi202.stdout:[client] 2023-06-22T17:03:16.391 INFO:teuthology.orchestra.run.smithi202.stdout: rgw cache enabled = true 2023-06-22T17:03:16.391 INFO:teuthology.orchestra.run.smithi202.stdout: rgw enable ops log = true 2023-06-22T17:03:16.391 INFO:teuthology.orchestra.run.smithi202.stdout: rgw enable usage log = true 2023-06-22T17:03:16.391 INFO:teuthology.orchestra.run.smithi202.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-22T17:03:16.392 INFO:teuthology.orchestra.run.smithi202.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-22T17:03:16.392 INFO:teuthology.orchestra.run.smithi202.stdout: client mount timeout = 600 2023-06-22T17:03:16.392 INFO:teuthology.orchestra.run.smithi202.stdout: debug client = 20 2023-06-22T17:03:16.392 INFO:teuthology.orchestra.run.smithi202.stdout: debug ms = 1 2023-06-22T17:03:16.393 INFO:teuthology.orchestra.run.smithi202.stdout: pid file = "" 2023-06-22T17:03:16.393 INFO:teuthology.orchestra.run.smithi202.stdout: rados mon op timeout = 900 2023-06-22T17:03:16.393 INFO:teuthology.orchestra.run.smithi202.stdout: rados osd op timeout = 900 2023-06-22T17:03:16.393 INFO:teuthology.orchestra.run.smithi202.stdout:[mon.a] 2023-06-22T17:03:16.394 INFO:teuthology.orchestra.run.smithi202.stdout:[mon.b] 2023-06-22T17:03:16.394 INFO:teuthology.orchestra.run.smithi202.stdout:[mon.c] 2023-06-22T17:03:16.394 INFO:teuthology.orchestra.run.smithi202.stdout:[mds.a] 2023-06-22T17:03:16.394 INFO:teuthology.orchestra.run.smithi202.stdout:[mds.c] 2023-06-22T17:03:16.395 INFO:teuthology.orchestra.run.smithi202.stdout:[mds.b] 2023-06-22T17:03:16.395 INFO:teuthology.orchestra.run.smithi202.stdout:[mds.d] 2023-06-22T17:03:16.395 INFO:teuthology.orchestra.run.smithi202.stdout:[mds] 2023-06-22T17:03:16.395 INFO:teuthology.orchestra.run.smithi202.stdout: debug mds = 20 2023-06-22T17:03:16.396 INFO:teuthology.orchestra.run.smithi202.stdout: debug ms = 1 2023-06-22T17:03:16.396 INFO:teuthology.orchestra.run.smithi202.stdout: mds debug frag = True 2023-06-22T17:03:16.396 INFO:teuthology.orchestra.run.smithi202.stdout: mds debug scatterstat = True 2023-06-22T17:03:16.397 INFO:teuthology.orchestra.run.smithi202.stdout: mds op complaint time = 180 2023-06-22T17:03:16.397 INFO:teuthology.orchestra.run.smithi202.stdout: mds verify scatter = True 2023-06-22T17:03:16.397 INFO:teuthology.orchestra.run.smithi202.stdout: osd op complaint time = 180 2023-06-22T17:03:16.397 INFO:teuthology.orchestra.run.smithi202.stdout: rados mon op timeout = 900 2023-06-22T17:03:16.397 INFO:teuthology.orchestra.run.smithi202.stdout: rados osd op timeout = 900 2023-06-22T17:03:16.398 INFO:tasks.ceph:Creating admin key on mon.a... 2023-06-22T17:03:16.398 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-06-22T17:03:16.451 INFO:tasks.ceph:Copying monmap to all nodes... 2023-06-22T17:03:16.452 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:16.452 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-06-22T17:03:16.467 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:16.468 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-06-22T17:03:16.522 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T17:03:16.523 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:16.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-22T17:03:16.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-22T17:03:16.598 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:16.598 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:16.654 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T17:03:16.655 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:16.655 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-22T17:03:16.656 DEBUG:teuthology.orchestra.run.smithi202:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-22T17:03:16.690 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:16.691 DEBUG:teuthology.orchestra.run.smithi202:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:16.745 INFO:tasks.ceph:Setting up mon nodes... 2023-06-22T17:03:16.745 INFO:tasks.ceph:Setting up mgr nodes... 2023-06-22T17:03:16.746 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-06-22T17:03:16.795 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-06-22T17:03:16.797 DEBUG:teuthology.orchestra.run.smithi202:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-06-22T17:03:16.858 INFO:teuthology.orchestra.run.smithi202.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-06-22T17:03:16.860 INFO:tasks.ceph:Setting up mds nodes... 2023-06-22T17:03:16.860 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-06-22T17:03:16.910 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-06-22T17:03:16.912 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-06-22T17:03:16.937 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-06-22T17:03:17.027 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-06-22T17:03:17.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-06-22T17:03:17.054 DEBUG:teuthology.orchestra.run.smithi202:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-06-22T17:03:17.103 INFO:teuthology.orchestra.run.smithi202.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-06-22T17:03:17.105 DEBUG:teuthology.orchestra.run.smithi202:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-06-22T17:03:17.129 DEBUG:teuthology.orchestra.run.smithi202:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-06-22T17:03:17.216 INFO:teuthology.orchestra.run.smithi202.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-06-22T17:03:17.218 DEBUG:teuthology.orchestra.run.smithi202:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-06-22T17:03:17.242 INFO:tasks.ceph_client:Setting up client nodes... 2023-06-22T17:03:17.242 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-22T17:03:17.282 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-06-22T17:03:17.295 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-06-22T17:03:17.374 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-06-22T17:03:17.386 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-06-22T17:03:17.424 INFO:teuthology.orchestra.run.smithi202.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-06-22T17:03:17.436 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-06-22T17:03:17.513 INFO:teuthology.orchestra.run.smithi202.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-06-22T17:03:17.525 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-06-22T17:03:17.525 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi202.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-22T17:03:17.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-06-22T17:03:17.550 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-22T17:03:17.551 INFO:tasks.ceph:role: osd.0 2023-06-22T17:03:17.551 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T17:03:17.551 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-22T17:03:17.615 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-22T17:03:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-22T17:03:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-22T17:03:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2023-06-22T17:03:17.616 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-22T17:03:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2023-06-22T17:03:17.616 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-22T17:03:17.616 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-22T17:03:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-22T17:03:17.617 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-22T17:03:17.638 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2023-06-22T17:03:17.639 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2023-06-22T17:03:17.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-06-22T17:03:17.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-06-22T17:03:17.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-06-22T17:03:17.854 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-22T17:03:17.854 INFO:tasks.ceph:role: osd.1 2023-06-22T17:03:17.854 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T17:03:17.855 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-22T17:03:17.919 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-22T17:03:17.919 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-22T17:03:17.920 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-22T17:03:17.920 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2023-06-22T17:03:17.920 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-22T17:03:17.921 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2023-06-22T17:03:17.921 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-22T17:03:17.921 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-22T17:03:17.922 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-22T17:03:17.922 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-22T17:03:17.941 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2023-06-22T17:03:17.943 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2023-06-22T17:03:17.943 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-06-22T17:03:18.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-06-22T17:03:18.090 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-06-22T17:03:18.155 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-22T17:03:18.156 INFO:tasks.ceph:role: osd.2 2023-06-22T17:03:18.156 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T17:03:18.156 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-22T17:03:18.220 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-22T17:03:18.220 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-22T17:03:18.221 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-22T17:03:18.221 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2023-06-22T17:03:18.221 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-22T17:03:18.222 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2023-06-22T17:03:18.222 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-22T17:03:18.222 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-22T17:03:18.222 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-22T17:03:18.223 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-22T17:03:18.242 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2023-06-22T17:03:18.244 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2023-06-22T17:03:18.244 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-06-22T17:03:18.322 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-06-22T17:03:18.392 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-06-22T17:03:18.456 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-22T17:03:18.457 INFO:tasks.ceph:role: osd.3 2023-06-22T17:03:18.458 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T17:03:18.458 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-22T17:03:18.521 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-22T17:03:18.522 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-22T17:03:18.522 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-22T17:03:18.522 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 2023-06-22T17:03:18.522 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-22T17:03:18.522 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2023-06-22T17:03:18.522 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-22T17:03:18.522 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-22T17:03:18.523 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-22T17:03:18.523 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-22T17:03:18.543 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2023-06-22T17:03:18.544 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2023-06-22T17:03:18.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-06-22T17:03:18.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-06-22T17:03:18.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:18.778+0000 7f4414fe3540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-06-22T17:03:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:18.778+0000 7f4414fe3540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-06-22T17:03:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:18.778+0000 7f4414fe3540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-22T17:03:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:18.778+0000 7f4414fe3540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-22T17:03:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:18.778+0000 7f4414fe3540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-22T17:03:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:18.778+0000 7f4414fe3540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-06-22T17:03:19.302 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:19.301+0000 7f4414fe3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:20.065 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:20.063+0000 7f4414fe3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:20.830 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:20.829+0000 7f4414fe3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:21.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-06-22T17:03:21.656 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:21.742+0000 7fb11aa9d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-06-22T17:03:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:21.742+0000 7fb11aa9d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-06-22T17:03:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:21.742+0000 7fb11aa9d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-22T17:03:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:21.742+0000 7fb11aa9d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-22T17:03:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:21.742+0000 7fb11aa9d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-22T17:03:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:21.744+0000 7fb11aa9d540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-06-22T17:03:22.267 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:22.265+0000 7fb11aa9d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:23.039 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:23.038+0000 7fb11aa9d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:23.800 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:23.800+0000 7fb11aa9d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:24.594 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-06-22T17:03:24.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:24.712 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:24.711+0000 7f9d54897540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-06-22T17:03:24.713 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:24.711+0000 7f9d54897540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-06-22T17:03:24.713 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:24.711+0000 7f9d54897540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-22T17:03:24.713 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:24.711+0000 7f9d54897540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-22T17:03:24.713 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:24.711+0000 7f9d54897540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-22T17:03:24.713 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:24.711+0000 7f9d54897540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-06-22T17:03:25.234 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:25.234+0000 7f9d54897540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:25.999+0000 7f9d54897540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:26.762+0000 7f9d54897540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:27.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-06-22T17:03:27.587 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:27.675 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:27.674+0000 7f1e50fe5540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-06-22T17:03:27.676 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:27.674+0000 7f1e50fe5540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-06-22T17:03:27.676 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:27.674+0000 7f1e50fe5540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-22T17:03:27.676 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:27.674+0000 7f1e50fe5540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-22T17:03:27.676 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:27.674+0000 7f1e50fe5540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-22T17:03:27.676 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:27.675+0000 7f1e50fe5540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-06-22T17:03:28.201 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:28.200+0000 7f1e50fe5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:28.969+0000 7f1e50fe5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:29.732+0000 7f1e50fe5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:30.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-06-22T17:03:30.560 DEBUG:teuthology.orchestra.run.smithi202:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-06-22T17:03:30.586 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-22T17:03:30.586 INFO:tasks.ceph:role: osd.4 2023-06-22T17:03:30.586 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T17:03:30.587 DEBUG:teuthology.orchestra.run.smithi202:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-22T17:03:30.660 INFO:teuthology.orchestra.run.smithi202.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-22T17:03:30.660 INFO:teuthology.orchestra.run.smithi202.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-22T17:03:30.660 INFO:teuthology.orchestra.run.smithi202.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-22T17:03:30.660 INFO:teuthology.orchestra.run.smithi202.stdout: = reflink=1 2023-06-22T17:03:30.660 INFO:teuthology.orchestra.run.smithi202.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-22T17:03:30.661 INFO:teuthology.orchestra.run.smithi202.stdout: = sunit=0 swidth=0 blks 2023-06-22T17:03:30.661 INFO:teuthology.orchestra.run.smithi202.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-22T17:03:30.661 INFO:teuthology.orchestra.run.smithi202.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-22T17:03:30.661 INFO:teuthology.orchestra.run.smithi202.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-22T17:03:30.661 INFO:teuthology.orchestra.run.smithi202.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-22T17:03:31.092 INFO:teuthology.orchestra.run.smithi202.stdout:Discarding blocks...Done. 2023-06-22T17:03:31.094 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi202.front.sepia.ceph.com -o noatime 2023-06-22T17:03:31.095 DEBUG:teuthology.orchestra.run.smithi202:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-06-22T17:03:31.256 DEBUG:teuthology.orchestra.run.smithi202:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-06-22T17:03:31.286 DEBUG:teuthology.orchestra.run.smithi202:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-06-22T17:03:31.350 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-22T17:03:31.350 INFO:tasks.ceph:role: osd.5 2023-06-22T17:03:31.350 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T17:03:31.351 DEBUG:teuthology.orchestra.run.smithi202:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-22T17:03:31.422 INFO:teuthology.orchestra.run.smithi202.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-22T17:03:31.422 INFO:teuthology.orchestra.run.smithi202.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-22T17:03:31.423 INFO:teuthology.orchestra.run.smithi202.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-22T17:03:31.423 INFO:teuthology.orchestra.run.smithi202.stdout: = reflink=1 2023-06-22T17:03:31.424 INFO:teuthology.orchestra.run.smithi202.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-22T17:03:31.424 INFO:teuthology.orchestra.run.smithi202.stdout: = sunit=0 swidth=0 blks 2023-06-22T17:03:31.424 INFO:teuthology.orchestra.run.smithi202.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-22T17:03:31.424 INFO:teuthology.orchestra.run.smithi202.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-22T17:03:31.424 INFO:teuthology.orchestra.run.smithi202.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-22T17:03:31.424 INFO:teuthology.orchestra.run.smithi202.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-22T17:03:31.839 INFO:teuthology.orchestra.run.smithi202.stdout:Discarding blocks...Done. 2023-06-22T17:03:31.841 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi202.front.sepia.ceph.com -o noatime 2023-06-22T17:03:31.841 DEBUG:teuthology.orchestra.run.smithi202:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-06-22T17:03:31.943 DEBUG:teuthology.orchestra.run.smithi202:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-06-22T17:03:31.971 DEBUG:teuthology.orchestra.run.smithi202:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-06-22T17:03:32.036 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-22T17:03:32.037 INFO:tasks.ceph:role: osd.6 2023-06-22T17:03:32.037 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T17:03:32.037 DEBUG:teuthology.orchestra.run.smithi202:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-22T17:03:32.107 INFO:teuthology.orchestra.run.smithi202.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-22T17:03:32.108 INFO:teuthology.orchestra.run.smithi202.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-22T17:03:32.108 INFO:teuthology.orchestra.run.smithi202.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-22T17:03:32.108 INFO:teuthology.orchestra.run.smithi202.stdout: = reflink=1 2023-06-22T17:03:32.108 INFO:teuthology.orchestra.run.smithi202.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-22T17:03:32.108 INFO:teuthology.orchestra.run.smithi202.stdout: = sunit=0 swidth=0 blks 2023-06-22T17:03:32.109 INFO:teuthology.orchestra.run.smithi202.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-22T17:03:32.109 INFO:teuthology.orchestra.run.smithi202.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-22T17:03:32.109 INFO:teuthology.orchestra.run.smithi202.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-22T17:03:32.109 INFO:teuthology.orchestra.run.smithi202.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-22T17:03:32.558 INFO:teuthology.orchestra.run.smithi202.stdout:Discarding blocks...Done. 2023-06-22T17:03:32.559 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi202.front.sepia.ceph.com -o noatime 2023-06-22T17:03:32.560 DEBUG:teuthology.orchestra.run.smithi202:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-06-22T17:03:32.711 DEBUG:teuthology.orchestra.run.smithi202:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-06-22T17:03:32.742 DEBUG:teuthology.orchestra.run.smithi202:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-06-22T17:03:32.806 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-22T17:03:32.807 INFO:tasks.ceph:role: osd.7 2023-06-22T17:03:32.807 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T17:03:32.807 DEBUG:teuthology.orchestra.run.smithi202:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-22T17:03:32.878 INFO:teuthology.orchestra.run.smithi202.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-22T17:03:32.878 INFO:teuthology.orchestra.run.smithi202.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-22T17:03:32.879 INFO:teuthology.orchestra.run.smithi202.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-22T17:03:32.879 INFO:teuthology.orchestra.run.smithi202.stdout: = reflink=1 2023-06-22T17:03:32.879 INFO:teuthology.orchestra.run.smithi202.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-22T17:03:32.879 INFO:teuthology.orchestra.run.smithi202.stdout: = sunit=0 swidth=0 blks 2023-06-22T17:03:32.879 INFO:teuthology.orchestra.run.smithi202.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-22T17:03:32.879 INFO:teuthology.orchestra.run.smithi202.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-22T17:03:32.879 INFO:teuthology.orchestra.run.smithi202.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-22T17:03:32.880 INFO:teuthology.orchestra.run.smithi202.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-22T17:03:33.320 INFO:teuthology.orchestra.run.smithi202.stdout:Discarding blocks...Done. 2023-06-22T17:03:33.322 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi202.front.sepia.ceph.com -o noatime 2023-06-22T17:03:33.322 DEBUG:teuthology.orchestra.run.smithi202:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-06-22T17:03:33.503 DEBUG:teuthology.orchestra.run.smithi202:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-06-22T17:03:33.535 DEBUG:teuthology.orchestra.run.smithi202:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:33.734 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:33.734+0000 7fabcd13a540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-06-22T17:03:33.735 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:33.734+0000 7fabcd13a540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-06-22T17:03:33.735 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:33.734+0000 7fabcd13a540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-22T17:03:33.735 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:33.734+0000 7fabcd13a540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-22T17:03:33.735 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:33.734+0000 7fabcd13a540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-22T17:03:33.735 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:33.735+0000 7fabcd13a540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-06-22T17:03:34.262 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:34.262+0000 7fabcd13a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:35.029 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:35.029+0000 7fabcd13a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:35.791 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:35.791+0000 7fabcd13a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:36.584 DEBUG:teuthology.orchestra.run.smithi202:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-06-22T17:03:36.613 DEBUG:teuthology.orchestra.run.smithi202:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:36.699 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:36.698+0000 7f2d36972540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-06-22T17:03:36.699 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:36.699+0000 7f2d36972540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-06-22T17:03:36.699 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:36.699+0000 7f2d36972540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-22T17:03:36.699 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:36.699+0000 7f2d36972540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-22T17:03:36.700 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:36.699+0000 7f2d36972540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-22T17:03:36.700 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:36.699+0000 7f2d36972540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-06-22T17:03:37.221 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:37.221+0000 7f2d36972540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:37.982 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:37.981+0000 7f2d36972540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:38.744 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:38.744+0000 7f2d36972540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:39.538 DEBUG:teuthology.orchestra.run.smithi202:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-06-22T17:03:39.602 DEBUG:teuthology.orchestra.run.smithi202:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:39.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:39.682+0000 7f0760db5540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-06-22T17:03:39.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:39.682+0000 7f0760db5540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-06-22T17:03:39.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:39.683+0000 7f0760db5540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-22T17:03:39.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:39.683+0000 7f0760db5540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-22T17:03:39.685 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:39.683+0000 7f0760db5540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-22T17:03:39.685 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:39.683+0000 7f0760db5540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-06-22T17:03:40.205 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:40.204+0000 7f0760db5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:40.967 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:40.965+0000 7f0760db5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:41.729 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:41.728+0000 7f0760db5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:42.522 DEBUG:teuthology.orchestra.run.smithi202:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-06-22T17:03:42.588 DEBUG:teuthology.orchestra.run.smithi202:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:42.671 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:42.670+0000 7f1a8499c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-06-22T17:03:42.672 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:42.670+0000 7f1a8499c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-06-22T17:03:42.672 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:42.670+0000 7f1a8499c540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-22T17:03:42.672 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:42.670+0000 7f1a8499c540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-22T17:03:42.672 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:42.670+0000 7f1a8499c540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-22T17:03:42.672 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:42.670+0000 7f1a8499c540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-06-22T17:03:43.191 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:43.190+0000 7f1a8499c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:43.957 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:43.955+0000 7f1a8499c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:44.322 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-22T17:03:44.325 DEBUG:teuthology.orchestra.run.smithi202:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-22T17:03:44.719 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:44.718+0000 7f1a8499c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-22T17:03:45.513 DEBUG:teuthology.orchestra.run.smithi202:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-06-22T17:03:45.538 INFO:tasks.ceph:Reading keys from all nodes... 2023-06-22T17:03:45.539 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:45.539 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-06-22T17:03:45.569 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:45.569 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-06-22T17:03:45.637 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:45.637 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-06-22T17:03:45.704 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:45.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-06-22T17:03:45.771 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:45.771 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-06-22T17:03:45.837 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:45.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-06-22T17:03:45.903 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:45.904 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-06-22T17:03:45.968 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:45.968 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-06-22T17:03:45.993 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:45.994 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-06-22T17:03:46.059 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:46.059 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-06-22T17:03:46.122 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:46.123 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-06-22T17:03:46.186 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:46.187 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-06-22T17:03:46.250 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:46.251 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-06-22T17:03:46.313 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:46.314 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-06-22T17:03:46.377 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:46.378 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-06-22T17:03:46.393 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:46.394 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-06-22T17:03:46.449 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:46.450 DEBUG:teuthology.orchestra.run.smithi202:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-06-22T17:03:46.464 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:46.464 DEBUG:teuthology.orchestra.run.smithi202:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-06-22T17:03:46.518 INFO:tasks.ceph:Adding keys to all mons... 2023-06-22T17:03:46.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-22T17:03:46.520 DEBUG:teuthology.orchestra.run.smithi202:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-22T17:03:46.561 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.y] 2023-06-22T17:03:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDUfpRkCWxlLxAAJBLspHXxpJPMVJ0Ii9aCFw== 2023-06-22T17:03:46.561 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2023-06-22T17:03:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDUfpRkantHNhAAdLH/k19DFQxLplIbZJb+Sg== 2023-06-22T17:03:46.562 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2023-06-22T17:03:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDVfpRkvZueARAA0e4TaPqBwVdGEfSiFPT1Mg== 2023-06-22T17:03:46.562 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2023-06-22T17:03:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDWfpRko2x/LhAA/iS9vg3kJ0HxKLWejX5vpA== 2023-06-22T17:03:46.562 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2023-06-22T17:03:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDZfpRktxldLBAAa999BWHa3nrHVskyItOqXg== 2023-06-22T17:03:46.563 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2023-06-22T17:03:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDcfpRkkhxwKhAAwhC/V3kmMdjf35GUgeA19Q== 2023-06-22T17:03:46.563 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2023-06-22T17:03:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDffpRkBLA+KBAAJyppPEJ9GH/yS+5Qb567Og== 2023-06-22T17:03:46.563 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2023-06-22T17:03:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDUfpRkQColMxAAFFDhaxXE15dSTJhmu4ofYQ== 2023-06-22T17:03:46.563 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2023-06-22T17:03:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDVfpRk1lQnBhAAxVFTXwhUakHpdrnu+xi6Jg== 2023-06-22T17:03:46.564 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2023-06-22T17:03:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDVfpRkakPoDBAA32+2heTWyBBLrZ1IQj9d5A== 2023-06-22T17:03:46.564 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.4] 2023-06-22T17:03:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDlfpRkRzzHKxAA6L2fl5TE+JsmpEGe+pi2XA== 2023-06-22T17:03:46.565 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.5] 2023-06-22T17:03:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDofpRkxlWqKRAAbPOIe7oCJ93Dc5P2CRwWJw== 2023-06-22T17:03:46.565 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.6] 2023-06-22T17:03:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDrfpRk5kjDKBAAiAtRdBFaVSq72fMPZ/u74w== 2023-06-22T17:03:46.566 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.7] 2023-06-22T17:03:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDufpRkz28DKBAABjKp2v2Nbc6r/eDJ8lSNDg== 2023-06-22T17:03:46.566 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2023-06-22T17:03:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDVfpRkwXjUEBAAN7jYM8XxHNSqiEG7Z415Gg== 2023-06-22T17:03:46.566 INFO:teuthology.orchestra.run.smithi047.stdout:[client.2] 2023-06-22T17:03:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDVfpRkQOpJFhAAy1ykSrBb/PBMBPUtLXiUZg== 2023-06-22T17:03:46.567 INFO:teuthology.orchestra.run.smithi047.stdout:[client.1] 2023-06-22T17:03:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDVfpRkoU5LGRAA6aYlduCrbPnCtxGIFi7cYg== 2023-06-22T17:03:46.567 INFO:teuthology.orchestra.run.smithi047.stdout:[client.3] 2023-06-22T17:03:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDVfpRkl36aHhAA2/6Tot95qULHZDD5FwzYmw== 2023-06-22T17:03:46.580 INFO:teuthology.orchestra.run.smithi202.stdout:[mgr.y] 2023-06-22T17:03:46.580 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDUfpRkCWxlLxAAJBLspHXxpJPMVJ0Ii9aCFw== 2023-06-22T17:03:46.581 INFO:teuthology.orchestra.run.smithi202.stdout:[mds.a] 2023-06-22T17:03:46.581 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDUfpRkantHNhAAdLH/k19DFQxLplIbZJb+Sg== 2023-06-22T17:03:46.581 INFO:teuthology.orchestra.run.smithi202.stdout:[mds.c] 2023-06-22T17:03:46.581 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDVfpRkvZueARAA0e4TaPqBwVdGEfSiFPT1Mg== 2023-06-22T17:03:46.581 INFO:teuthology.orchestra.run.smithi202.stdout:[osd.0] 2023-06-22T17:03:46.581 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDWfpRko2x/LhAA/iS9vg3kJ0HxKLWejX5vpA== 2023-06-22T17:03:46.581 INFO:teuthology.orchestra.run.smithi202.stdout:[osd.1] 2023-06-22T17:03:46.582 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDZfpRktxldLBAAa999BWHa3nrHVskyItOqXg== 2023-06-22T17:03:46.582 INFO:teuthology.orchestra.run.smithi202.stdout:[osd.2] 2023-06-22T17:03:46.582 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDcfpRkkhxwKhAAwhC/V3kmMdjf35GUgeA19Q== 2023-06-22T17:03:46.582 INFO:teuthology.orchestra.run.smithi202.stdout:[osd.3] 2023-06-22T17:03:46.582 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDffpRkBLA+KBAAJyppPEJ9GH/yS+5Qb567Og== 2023-06-22T17:03:46.582 INFO:teuthology.orchestra.run.smithi202.stdout:[mgr.x] 2023-06-22T17:03:46.582 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDUfpRkQColMxAAFFDhaxXE15dSTJhmu4ofYQ== 2023-06-22T17:03:46.582 INFO:teuthology.orchestra.run.smithi202.stdout:[mds.b] 2023-06-22T17:03:46.583 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDVfpRk1lQnBhAAxVFTXwhUakHpdrnu+xi6Jg== 2023-06-22T17:03:46.583 INFO:teuthology.orchestra.run.smithi202.stdout:[mds.d] 2023-06-22T17:03:46.583 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDVfpRkakPoDBAA32+2heTWyBBLrZ1IQj9d5A== 2023-06-22T17:03:46.583 INFO:teuthology.orchestra.run.smithi202.stdout:[osd.4] 2023-06-22T17:03:46.583 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDlfpRkRzzHKxAA6L2fl5TE+JsmpEGe+pi2XA== 2023-06-22T17:03:46.583 INFO:teuthology.orchestra.run.smithi202.stdout:[osd.5] 2023-06-22T17:03:46.583 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDofpRkxlWqKRAAbPOIe7oCJ93Dc5P2CRwWJw== 2023-06-22T17:03:46.583 INFO:teuthology.orchestra.run.smithi202.stdout:[osd.6] 2023-06-22T17:03:46.584 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDrfpRk5kjDKBAAiAtRdBFaVSq72fMPZ/u74w== 2023-06-22T17:03:46.584 INFO:teuthology.orchestra.run.smithi202.stdout:[osd.7] 2023-06-22T17:03:46.584 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDufpRkz28DKBAABjKp2v2Nbc6r/eDJ8lSNDg== 2023-06-22T17:03:46.584 INFO:teuthology.orchestra.run.smithi202.stdout:[client.0] 2023-06-22T17:03:46.584 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDVfpRkwXjUEBAAN7jYM8XxHNSqiEG7Z415Gg== 2023-06-22T17:03:46.584 INFO:teuthology.orchestra.run.smithi202.stdout:[client.2] 2023-06-22T17:03:46.584 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDVfpRkQOpJFhAAy1ykSrBb/PBMBPUtLXiUZg== 2023-06-22T17:03:46.585 INFO:teuthology.orchestra.run.smithi202.stdout:[client.1] 2023-06-22T17:03:46.585 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDVfpRkoU5LGRAA6aYlduCrbPnCtxGIFi7cYg== 2023-06-22T17:03:46.585 INFO:teuthology.orchestra.run.smithi202.stdout:[client.3] 2023-06-22T17:03:46.585 INFO:teuthology.orchestra.run.smithi202.stdout: key = AQDVfpRkl36aHhAA2/6Tot95qULHZDD5FwzYmw== 2023-06-22T17:03:46.586 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-22T17:03:46.612 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-22T17:03:46.673 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-22T17:03:46.677 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-22T17:03:46.727 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-22T17:03:46.730 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-22T17:03:46.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:46.782 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:46.830 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:46.832 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:46.879 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:46.881 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:46.928 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:46.930 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:46.976 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-22T17:03:46.979 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-22T17:03:47.026 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-22T17:03:47.028 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-22T17:03:47.078 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-22T17:03:47.080 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-22T17:03:47.130 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:47.132 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:47.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:47.181 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:47.229 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:47.231 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:47.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:47.280 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-22T17:03:47.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-22T17:03:47.330 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-22T17:03:47.379 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-22T17:03:47.381 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-22T17:03:47.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-22T17:03:47.435 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-22T17:03:47.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-22T17:03:47.487 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-22T17:03:47.537 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-06-22T17:03:47.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-06-22T17:03:47.562 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-22T17:03:47.655 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-06-22T17:03:47.679 DEBUG:teuthology.orchestra.run.smithi202:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-06-22T17:03:47.704 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-22T17:03:47.795 DEBUG:teuthology.orchestra.run.smithi202:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-06-22T17:03:47.820 DEBUG:teuthology.orchestra.run.smithi202:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-06-22T17:03:47.884 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-22T17:03:47.977 DEBUG:teuthology.orchestra.run.smithi202:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-06-22T17:03:48.001 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:48.004 DEBUG:teuthology.orchestra.run.smithi202:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-22T17:03:48.055 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-06-22T17:03:48.056 INFO:tasks.ceph.mon.a:Restarting daemon 2023-06-22T17:03:48.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-06-22T17:03:48.059 INFO:tasks.ceph.mon.a:Started 2023-06-22T17:03:48.060 INFO:tasks.ceph.mon.b:Restarting daemon 2023-06-22T17:03:48.060 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-06-22T17:03:48.097 INFO:tasks.ceph.mon.b:Started 2023-06-22T17:03:48.097 INFO:tasks.ceph.mon.c:Restarting daemon 2023-06-22T17:03:48.098 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-06-22T17:03:48.099 INFO:tasks.ceph.mon.c:Started 2023-06-22T17:03:48.100 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-06-22T17:03:48.100 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-06-22T17:03:48.100 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-06-22T17:03:48.102 INFO:tasks.ceph.mgr.y:Started 2023-06-22T17:03:48.103 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-06-22T17:03:48.103 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-06-22T17:03:48.105 INFO:tasks.ceph.mgr.x:Started 2023-06-22T17:03:48.105 INFO:tasks.ceph:Setting crush tunables to default 2023-06-22T17:03:48.106 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd crush tunables default 2023-06-22T17:03:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.226+0000 7f95890aa700 1 Processor -- start 2023-06-22T17:03:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.226+0000 7f95890aa700 1 -- start start 2023-06-22T17:03:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f95890aa700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 0x7f95841596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f95890aa700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95840cb7d0 con 0x7f9584159310 2023-06-22T17:03:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f95890aa700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95840c72b0 con 0x7f95840b23d0 2023-06-22T17:03:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f95890aa700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95840b2e10 con 0x7f95840b07f0 2023-06-22T17:03:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f9583fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9584159310 0x7f958415dc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60948/0 (socket says 172.21.15.47:60948) 2023-06-22T17:03:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f9583fff700 1 -- 172.21.15.47:0/3921036563 learned_addr learned my addr 172.21.15.47:0/3921036563 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:03:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f95888a9700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 0x7f95841596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f95888a9700 1 -- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 msgr2=0x7f95841596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-22T17:03:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f95888a9700 1 -- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 msgr2=0x7f95841596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-22T17:03:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f95888a9700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 0x7f95841596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-22T17:03:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.227+0000 7f95888a9700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 0x7f95841596e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-22T17:03:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764795883 0 0) 0x7f95840c72b0 con 0x7f95840b23d0 2023-06-22T17:03:48.411 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 --> v1:172.21.15.202:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9570003570 con 0x7f95840b23d0 2023-06-22T17:03:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 <== mon.1 v1:172.21.15.202:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 145728400 0 0) 0x7f9570003570 con 0x7f95840b23d0 2023-06-22T17:03:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 msgr2=0x7f95841596e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-22T17:03:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 0x7f95841596e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 >> v1:172.21.15.47:6789/0 conn(0x7f9584159310 legacy=0x7f958415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 --> v1:172.21.15.202:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95840b2850 con 0x7f95840b23d0 2023-06-22T17:03:48.412 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 <== mon.1 v1:172.21.15.202:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 763095103 0 0) 0x7f956c0014b0 con 0x7f95840b23d0 2023-06-22T17:03:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 >> v1:172.21.15.202:6789/0 conn(0x7f95840b23d0 legacy=0x7f95840b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 0x7f9570003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700040e0 0x7f9570004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.409+0000 7f9582ffd700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f9570004da0 0x7f9570005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840b2e10 con 0x7f95700040e0 2023-06-22T17:03:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957000bca0 con 0x7f9570004da0 2023-06-22T17:03:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957000be60 con 0x7f95840b07f0 2023-06-22T17:03:48.413 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f95837fe700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700040e0 0x7f9570004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:48.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f9583fff700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 0x7f9570003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:48.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f95888a9700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f9570004da0 0x7f9570005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:48.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f95837fe700 1 -- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 msgr2=0x7f9570003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:48.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f95837fe700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 0x7f9570003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:48.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f95837fe700 1 -- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f9570004da0 msgr2=0x7f9570005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:48.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f95837fe700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f9570004da0 0x7f9570005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:48.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f95837fe700 1 -- 172.21.15.47:0/3921036563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570003570 con 0x7f95700040e0 2023-06-22T17:03:48.415 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f9583fff700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f95840b07f0 0x7f9570003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T17:03:48.415 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.410+0000 7f95837fe700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700040e0 0x7f9570004810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c160 tx=0x7f956c002e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d6422af1cee266 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:48.416 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.412+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c00ca90 con 0x7f95700040e0 2023-06-22T17:03:48.416 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.412+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c00cbf0 con 0x7f95700040e0 2023-06-22T17:03:48.416 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.412+0000 7f9582ffd700 1 -- 172.21.15.47:0/3921036563 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c006820 con 0x7f95700040e0 2023-06-22T17:03:48.416 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.412+0000 7f95890aa700 1 -- 172.21.15.47:0/3921036563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700040e0 msgr2=0x7f9570004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:48.416 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.412+0000 7f95890aa700 1 --2- 172.21.15.47:0/3921036563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95700040e0 0x7f9570004810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c160 tx=0x7f956c002e20 comp rx=0 tx=0).stop 2023-06-22T17:03:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.412+0000 7f95888a9700 1 -- 172.21.15.47:0/3921036563 reap_dead start 2023-06-22T17:03:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.412+0000 7f95890aa700 1 -- 172.21.15.47:0/3921036563 shutdown_connections 2023-06-22T17:03:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.412+0000 7f95890aa700 1 -- 172.21.15.47:0/3921036563 >> 172.21.15.47:0/3921036563 conn(0x7f9584070290 msgr2=0x7f9584070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.412+0000 7f95890aa700 1 -- 172.21.15.47:0/3921036563 shutdown_connections 2023-06-22T17:03:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.412+0000 7f95890aa700 1 -- 172.21.15.47:0/3921036563 wait complete. 2023-06-22T17:03:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.414+0000 7f95890aa700 1 Processor -- start 2023-06-22T17:03:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.414+0000 7f95890aa700 1 -- start start 2023-06-22T17:03:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f95890aa700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f95700040e0 0x7f95841575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f95890aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f958415b900 0x7f95841596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f95890aa700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9584159c20 0x7f95840c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f95890aa700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840c68a0 con 0x7f958415b900 2023-06-22T17:03:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f95890aa700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840c8950 con 0x7f95700040e0 2023-06-22T17:03:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f95890aa700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958415a070 con 0x7f9584159c20 2023-06-22T17:03:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f9583fff700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f95700040e0 0x7f95841575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f9583fff700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f95700040e0 0x7f95841575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3300/0 says I am v2:172.21.15.47:41454/0 (socket says 172.21.15.47:41454) 2023-06-22T17:03:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f9583fff700 1 -- 172.21.15.47:0/2030759731 learned_addr learned my addr 172.21.15.47:0/2030759731 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:03:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f95888a9700 1 --2- 172.21.15.47:0/2030759731 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9584159c20 0x7f95840c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f9583fff700 1 -- 172.21.15.47:0/2030759731 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9584159c20 msgr2=0x7f95840c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f9583fff700 1 --2- 172.21.15.47:0/2030759731 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9584159c20 0x7f95840c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f9583fff700 1 -- 172.21.15.47:0/2030759731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f958415b900 msgr2=0x7f95841596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f9583fff700 1 --2- 172.21.15.47:0/2030759731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f958415b900 0x7f95841596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.415+0000 7f9583fff700 1 -- 172.21.15.47:0/2030759731 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584157b00 con 0x7f95700040e0 2023-06-22T17:03:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.416+0000 7f9583fff700 1 --2- 172.21.15.47:0/2030759731 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f95700040e0 0x7f95841575c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95740073c0 tx=0x7f9574007de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.416+0000 7f957bfff700 1 -- 172.21.15.47:0/2030759731 <== mon.1 v2:172.21.15.202:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740048e0 con 0x7f95700040e0 2023-06-22T17:03:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.416+0000 7f957bfff700 1 -- 172.21.15.47:0/2030759731 <== mon.1 v2:172.21.15.202:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957400f8d0 con 0x7f95700040e0 2023-06-22T17:03:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.416+0000 7f95890aa700 1 -- 172.21.15.47:0/2030759731 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95840c65c0 con 0x7f95700040e0 2023-06-22T17:03:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.416+0000 7f957bfff700 1 -- 172.21.15.47:0/2030759731 <== mon.1 v2:172.21.15.202:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957400fc70 con 0x7f95700040e0 2023-06-22T17:03:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.416+0000 7f95890aa700 1 -- 172.21.15.47:0/2030759731 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95840c85e0 con 0x7f95700040e0 2023-06-22T17:03:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.416+0000 7f957bfff700 1 -- 172.21.15.47:0/2030759731 <== mon.1 v2:172.21.15.202:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f957401f6c0 con 0x7f95700040e0 2023-06-22T17:03:48.422 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.416+0000 7f957bfff700 1 -- 172.21.15.47:0/2030759731 <== mon.1 v2:172.21.15.202:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f957401e790 con 0x7f95700040e0 2023-06-22T17:03:48.422 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.417+0000 7f95890aa700 1 -- 172.21.15.47:0/2030759731 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9584067f00 con 0x7f95700040e0 2023-06-22T17:03:48.422 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.419+0000 7f957bfff700 1 -- 172.21.15.47:0/2030759731 <== mon.1 v2:172.21.15.202:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f9574024060 con 0x7f95700040e0 2023-06-22T17:03:48.497 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.496+0000 7f95890aa700 1 -- 172.21.15.47:0/2030759731 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f958404b970 con 0x7f95700040e0 2023-06-22T17:03:48.500 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.499+0000 7f957bfff700 1 -- 172.21.15.47:0/2030759731 <== mon.1 v2:172.21.15.202:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f957401a070 con 0x7f95700040e0 2023-06-22T17:03:48.501 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.500+0000 7f95890aa700 1 -- 172.21.15.47:0/2030759731 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f95700040e0 msgr2=0x7f95841575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:48.501 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.500+0000 7f95890aa700 1 --2- 172.21.15.47:0/2030759731 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f95700040e0 0x7f95841575c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95740073c0 tx=0x7f9574007de0 comp rx=0 tx=0).stop 2023-06-22T17:03:48.501 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.500+0000 7f95890aa700 1 -- 172.21.15.47:0/2030759731 shutdown_connections 2023-06-22T17:03:48.501 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.500+0000 7f95890aa700 1 --2- 172.21.15.47:0/2030759731 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9584159c20 0x7f95840c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:48.502 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.500+0000 7f95890aa700 1 --2- 172.21.15.47:0/2030759731 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f95700040e0 0x7f95841575c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:48.502 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.500+0000 7f95890aa700 1 --2- 172.21.15.47:0/2030759731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f958415b900 0x7f95841596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:48.502 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.500+0000 7f95890aa700 1 -- 172.21.15.47:0/2030759731 >> 172.21.15.47:0/2030759731 conn(0x7f9584070290 msgr2=0x7f95840ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:48.502 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.500+0000 7f95890aa700 1 -- 172.21.15.47:0/2030759731 shutdown_connections 2023-06-22T17:03:48.502 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:48.500+0000 7f95890aa700 1 -- 172.21.15.47:0/2030759731 wait complete. 2023-06-22T17:03:48.503 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2023-06-22T17:03:48.513 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-06-22T17:03:48.513 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:48.514 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-06-22T17:03:48.539 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:48.539 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-06-22T17:03:48.603 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:48.604 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-06-22T17:03:48.668 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2023-06-22T17:03:48.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-06-22T17:03:48.733 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:48.734 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-06-22T17:03:48.758 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:48.758 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-06-22T17:03:48.823 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:48.823 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-06-22T17:03:48.887 DEBUG:teuthology.orchestra.run.smithi202:> set -ex 2023-06-22T17:03:48.887 DEBUG:teuthology.orchestra.run.smithi202:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-06-22T17:03:48.951 DEBUG:teuthology.orchestra.run.smithi202:> sudo ceph --cluster ceph osd new 191ca9cf-dc97-4325-b63b-a3e6a66b71b1 0 2023-06-22T17:03:49.109 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.107+0000 7f6d8dc72700 1 Processor -- start 2023-06-22T17:03:49.109 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.107+0000 7f6d8dc72700 1 -- start start 2023-06-22T17:03:49.111 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.108+0000 7f6d8dc72700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153a80 0x7f6d880d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.111 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.108+0000 7f6d8dc72700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d880b2010 0x7f6d880b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.111 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.108+0000 7f6d8dc72700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d880cedd0 con 0x7f6d88153e50 2023-06-22T17:03:49.111 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.108+0000 7f6d8dc72700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880d14c0 con 0x7f6d880b2010 2023-06-22T17:03:49.112 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.108+0000 7f6d8dc72700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880b29d0 con 0x7f6d88153a80 2023-06-22T17:03:49.112 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.108+0000 7f6d877fe700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153a80 0x7f6d880d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.112 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.108+0000 7f6d7ffff700 1 --1- >> v1:172.21.15.202:6789/0 conn(0x7f6d88153e50 0x7f6d8815a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.202:6789/0 says I am v1:172.21.15.202:51956/0 (socket says 172.21.15.202:51956) 2023-06-22T17:03:49.112 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.108+0000 7f6d7ffff700 1 -- 172.21.15.202:0/1857803849 learned_addr learned my addr 172.21.15.202:0/1857803849 (peer_addr_for_me v1:172.21.15.202:0/0) 2023-06-22T17:03:49.112 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.108+0000 7f6d87fff700 1 --2- 172.21.15.202:0/1857803849 >> v2:172.21.15.47:3300/0 conn(0x7f6d880b2010 0x7f6d880b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.112 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.108+0000 7f6d86ffd700 1 -- 172.21.15.202:0/1857803849 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 860583901 0 0) 0x7f6d880cedd0 con 0x7f6d88153e50 2023-06-22T17:03:49.112 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.109+0000 7f6d877fe700 1 -- 172.21.15.202:0/1857803849 >> v2:172.21.15.47:3300/0 conn(0x7f6d880b2010 msgr2=0x7f6d880b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:49.113 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.109+0000 7f6d877fe700 1 --2- 172.21.15.202:0/1857803849 >> v2:172.21.15.47:3300/0 conn(0x7f6d880b2010 0x7f6d880b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.113 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.109+0000 7f6d877fe700 1 -- 172.21.15.202:0/1857803849 >> v1:172.21.15.202:6789/0 conn(0x7f6d88153e50 legacy=0x7f6d8815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.113 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.109+0000 7f6d877fe700 1 -- 172.21.15.202:0/1857803849 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8815aed0 con 0x7f6d88153a80 2023-06-22T17:03:49.113 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.109+0000 7f6d877fe700 1 --2- 172.21.15.202:0/1857803849 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153a80 0x7f6d880d0590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d780098e0 tx=0x7f6d780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54df6d42dd803c06 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:49.113 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.109+0000 7f6d86ffd700 1 -- 172.21.15.202:0/1857803849 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800b640 con 0x7f6d88153a80 2023-06-22T17:03:49.113 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.109+0000 7f6d86ffd700 1 -- 172.21.15.202:0/1857803849 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7800b7a0 con 0x7f6d88153a80 2023-06-22T17:03:49.113 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.109+0000 7f6d86ffd700 1 -- 172.21.15.202:0/1857803849 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800bb40 con 0x7f6d88153a80 2023-06-22T17:03:49.113 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.110+0000 7f6d8dc72700 1 -- 172.21.15.202:0/1857803849 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153a80 msgr2=0x7f6d880d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.114 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.110+0000 7f6d8dc72700 1 --2- 172.21.15.202:0/1857803849 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153a80 0x7f6d880d0590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d780098e0 tx=0x7f6d780091a0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.114 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.110+0000 7f6d8dc72700 1 -- 172.21.15.202:0/1857803849 shutdown_connections 2023-06-22T17:03:49.114 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.110+0000 7f6d8dc72700 1 --2- 172.21.15.202:0/1857803849 >> v2:172.21.15.47:3300/0 conn(0x7f6d880b2010 0x7f6d880b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.114 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.110+0000 7f6d8dc72700 1 --2- 172.21.15.202:0/1857803849 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153a80 0x7f6d880d0590 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.114 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.110+0000 7f6d8dc72700 1 -- 172.21.15.202:0/1857803849 >> 172.21.15.202:0/1857803849 conn(0x7f6d88070290 msgr2=0x7f6d88070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:49.114 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.110+0000 7f6d8dc72700 1 -- 172.21.15.202:0/1857803849 shutdown_connections 2023-06-22T17:03:49.114 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.110+0000 7f6d8dc72700 1 -- 172.21.15.202:0/1857803849 wait complete. 2023-06-22T17:03:49.116 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.111+0000 7f6d8dc72700 1 Processor -- start 2023-06-22T17:03:49.116 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.111+0000 7f6d8dc72700 1 -- start start 2023-06-22T17:03:49.116 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.111+0000 7f6d8dc72700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d880b2010 0x7f6d8816ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.117 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.111+0000 7f6d8dc72700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f6d88153a80 0x7f6d8816ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.117 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.111+0000 7f6d8dc72700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153e50 0x7f6d88152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.117 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.111+0000 7f6d8dc72700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d88156380 con 0x7f6d880b2010 2023-06-22T17:03:49.117 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d8dc72700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8816a8a0 con 0x7f6d88153a80 2023-06-22T17:03:49.117 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d8dc72700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d881565d0 con 0x7f6d88153e50 2023-06-22T17:03:49.117 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d87fff700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153e50 0x7f6d88152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.117 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d87fff700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153e50 0x7f6d88152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3301/0 says I am v2:172.21.15.202:55070/0 (socket says 172.21.15.202:55070) 2023-06-22T17:03:49.117 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d87fff700 1 -- 172.21.15.202:0/3360249331 learned_addr learned my addr 172.21.15.202:0/3360249331 (peer_addr_for_me v2:172.21.15.202:0/0) 2023-06-22T17:03:49.118 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d7ffff700 1 --2- 172.21.15.202:0/3360249331 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f6d88153a80 0x7f6d8816ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.118 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d877fe700 1 --2- 172.21.15.202:0/3360249331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d880b2010 0x7f6d8816ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.118 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d87fff700 1 -- 172.21.15.202:0/3360249331 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f6d88153a80 msgr2=0x7f6d8816ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.118 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d87fff700 1 --2- 172.21.15.202:0/3360249331 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f6d88153a80 0x7f6d8816ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.118 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d87fff700 1 -- 172.21.15.202:0/3360249331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d880b2010 msgr2=0x7f6d8816ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.118 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d87fff700 1 --2- 172.21.15.202:0/3360249331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d880b2010 0x7f6d8816ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.118 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d87fff700 1 -- 172.21.15.202:0/3360249331 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78009420 con 0x7f6d88153e50 2023-06-22T17:03:49.119 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d87fff700 1 --2- 172.21.15.202:0/3360249331 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153e50 0x7f6d88152ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009df0 tx=0x7f6d8000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:49.119 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d7f7fe700 1 -- 172.21.15.202:0/3360249331 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000caa0 con 0x7f6d88153e50 2023-06-22T17:03:49.119 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d7f7fe700 1 -- 172.21.15.202:0/3360249331 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8000f040 con 0x7f6d88153e50 2023-06-22T17:03:49.119 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d7f7fe700 1 -- 172.21.15.202:0/3360249331 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000ce70 con 0x7f6d88153e50 2023-06-22T17:03:49.119 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d8dc72700 1 -- 172.21.15.202:0/3360249331 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d881568b0 con 0x7f6d88153e50 2023-06-22T17:03:49.119 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.112+0000 7f6d8dc72700 1 -- 172.21.15.202:0/3360249331 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d881531e0 con 0x7f6d88153e50 2023-06-22T17:03:49.119 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.113+0000 7f6d7f7fe700 1 -- 172.21.15.202:0/3360249331 <== mon.2 v2:172.21.15.202:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6d80014690 con 0x7f6d88153e50 2023-06-22T17:03:49.119 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.113+0000 7f6d7f7fe700 1 -- 172.21.15.202:0/3360249331 <== mon.2 v2:172.21.15.202:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6d8001d880 con 0x7f6d88153e50 2023-06-22T17:03:49.120 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.113+0000 7f6d8dc72700 1 -- 172.21.15.202:0/3360249331 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8804c1b0 con 0x7f6d88153e50 2023-06-22T17:03:49.120 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.115+0000 7f6d7f7fe700 1 -- 172.21.15.202:0/3360249331 <== mon.2 v2:172.21.15.202:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f6d8003b6e0 con 0x7f6d88153e50 2023-06-22T17:03:49.199 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.198+0000 7f6d8dc72700 1 -- 172.21.15.202:0/3360249331 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "osd new", "uuid": "191ca9cf-dc97-4325-b63b-a3e6a66b71b1", "id": 0} v 0) v1 -- 0x7f6d88156dd0 con 0x7f6d88153e50 2023-06-22T17:03:49.202 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.201+0000 7f6d7f7fe700 1 -- 172.21.15.202:0/3360249331 <== mon.2 v2:172.21.15.202:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "191ca9cf-dc97-4325-b63b-a3e6a66b71b1", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6d88156dd0 con 0x7f6d88153e50 2023-06-22T17:03:49.203 INFO:teuthology.orchestra.run.smithi202.stdout:0 2023-06-22T17:03:49.204 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.202+0000 7f6d8dc72700 1 -- 172.21.15.202:0/3360249331 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153e50 msgr2=0x7f6d88152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.204 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.202+0000 7f6d8dc72700 1 --2- 172.21.15.202:0/3360249331 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153e50 0x7f6d88152ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009df0 tx=0x7f6d8000b040 comp rx=0 tx=0).stop 2023-06-22T17:03:49.204 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.202+0000 7f6d8dc72700 1 -- 172.21.15.202:0/3360249331 shutdown_connections 2023-06-22T17:03:49.204 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.202+0000 7f6d8dc72700 1 --2- 172.21.15.202:0/3360249331 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f6d88153e50 0x7f6d88152ca0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.204 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.202+0000 7f6d8dc72700 1 --2- 172.21.15.202:0/3360249331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d880b2010 0x7f6d8816ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.204 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.202+0000 7f6d8dc72700 1 --2- 172.21.15.202:0/3360249331 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f6d88153a80 0x7f6d8816ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.204 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.202+0000 7f6d8dc72700 1 -- 172.21.15.202:0/3360249331 >> 172.21.15.202:0/3360249331 conn(0x7f6d88070290 msgr2=0x7f6d8815c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:49.205 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.202+0000 7f6d8dc72700 1 -- 172.21.15.202:0/3360249331 shutdown_connections 2023-06-22T17:03:49.205 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.202+0000 7f6d8dc72700 1 -- 172.21.15.202:0/3360249331 wait complete. 2023-06-22T17:03:49.215 DEBUG:teuthology.orchestra.run.smithi202:> sudo ceph --cluster ceph osd new d426e8cd-b1e6-40a3-9e65-cf52fac95a21 1 2023-06-22T17:03:49.330 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.329+0000 7fa5cda5d700 1 Processor -- start 2023-06-22T17:03:49.331 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.329+0000 7fa5cda5d700 1 -- start start 2023-06-22T17:03:49.333 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5cda5d700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.333 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5cda5d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c80cb1b0 con 0x7fa5c8156ff0 2023-06-22T17:03:49.333 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5cda5d700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c815ab30 con 0x7fa5c80d0a40 2023-06-22T17:03:49.333 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5cda5d700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c815d6a0 con 0x7fa5c815d160 2023-06-22T17:03:49.333 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5c77fe700 1 --1- >> v1:172.21.15.202:6789/0 conn(0x7fa5c80d0a40 0x7fa5c80cb640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.202:6789/0 says I am v1:172.21.15.202:51966/0 (socket says 172.21.15.202:51966) 2023-06-22T17:03:49.333 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5c67fc700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.334 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5c77fe700 1 -- 172.21.15.202:0/659435559 learned_addr learned my addr 172.21.15.202:0/659435559 (peer_addr_for_me v1:172.21.15.202:0/0) 2023-06-22T17:03:49.334 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5c5ffb700 1 -- 172.21.15.202:0/659435559 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119963817 0 0) 0x7fa5c815ab30 con 0x7fa5c80d0a40 2023-06-22T17:03:49.334 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5c5ffb700 1 -- 172.21.15.202:0/659435559 --> v1:172.21.15.202:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b0003570 con 0x7fa5c80d0a40 2023-06-22T17:03:49.334 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5c67fc700 1 -- 172.21.15.202:0/659435559 >> v1:172.21.15.202:6789/0 conn(0x7fa5c80d0a40 legacy=0x7fa5c80cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.334 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5c67fc700 1 -- 172.21.15.202:0/659435559 >> v1:172.21.15.47:6789/0 conn(0x7fa5c8156ff0 legacy=0x7fa5c80d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.334 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.330+0000 7fa5c67fc700 1 -- 172.21.15.202:0/659435559 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c815d980 con 0x7fa5c815d160 2023-06-22T17:03:49.334 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5c67fc700 1 --2- 172.21.15.202:0/659435559 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c815a5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009910 tx=0x7fa5b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff034f76e8a2b51d server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:49.335 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5c5ffb700 1 -- 172.21.15.202:0/659435559 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b800b680 con 0x7fa5c815d160 2023-06-22T17:03:49.335 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5c5ffb700 1 -- 172.21.15.202:0/659435559 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b800b7e0 con 0x7fa5c815d160 2023-06-22T17:03:49.335 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5c5ffb700 1 -- 172.21.15.202:0/659435559 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b800bba0 con 0x7fa5c815d160 2023-06-22T17:03:49.335 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5cda5d700 1 -- 172.21.15.202:0/659435559 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 msgr2=0x7fa5c815a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.335 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5cda5d700 1 --2- 172.21.15.202:0/659435559 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c815a5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009910 tx=0x7fa5b80091a0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.335 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5cda5d700 1 -- 172.21.15.202:0/659435559 shutdown_connections 2023-06-22T17:03:49.335 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5cda5d700 1 --2- 172.21.15.202:0/659435559 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c815a5f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.335 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5cda5d700 1 -- 172.21.15.202:0/659435559 >> 172.21.15.202:0/659435559 conn(0x7fa5c8070290 msgr2=0x7fa5c8070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:49.336 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5cda5d700 1 -- 172.21.15.202:0/659435559 shutdown_connections 2023-06-22T17:03:49.336 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.331+0000 7fa5cda5d700 1 -- 172.21.15.202:0/659435559 wait complete. 2023-06-22T17:03:49.337 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5cda5d700 1 Processor -- start 2023-06-22T17:03:49.338 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5cda5d700 1 -- start start 2023-06-22T17:03:49.338 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5cda5d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c80d0a40 0x7fa5c80b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.338 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5cda5d700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa5c8156ff0 0x7fa5c80c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.338 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5cda5d700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c8149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.338 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5cda5d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80c7ae0 con 0x7fa5c80d0a40 2023-06-22T17:03:49.338 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5cda5d700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80c7cd0 con 0x7fa5c8156ff0 2023-06-22T17:03:49.338 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5cda5d700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80c7fb0 con 0x7fa5c815d160 2023-06-22T17:03:49.339 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5c77fe700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c8149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.339 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5c67fc700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa5c8156ff0 0x7fa5c80c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.339 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.333+0000 7fa5c77fe700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c8149010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3301/0 says I am v2:172.21.15.202:55092/0 (socket says 172.21.15.202:55092) 2023-06-22T17:03:49.339 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5c77fe700 1 -- 172.21.15.202:0/764382500 learned_addr learned my addr 172.21.15.202:0/764382500 (peer_addr_for_me v2:172.21.15.202:0/0) 2023-06-22T17:03:49.339 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5c6ffd700 1 --2- 172.21.15.202:0/764382500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c80d0a40 0x7fa5c80b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.339 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5c77fe700 1 -- 172.21.15.202:0/764382500 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa5c8156ff0 msgr2=0x7fa5c80c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.339 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5c77fe700 1 --2- 172.21.15.202:0/764382500 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa5c8156ff0 0x7fa5c80c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.340 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5c77fe700 1 -- 172.21.15.202:0/764382500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c80d0a40 msgr2=0x7fa5c80b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.340 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5c77fe700 1 --2- 172.21.15.202:0/764382500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c80d0a40 0x7fa5c80b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.340 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5c77fe700 1 -- 172.21.15.202:0/764382500 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b8009420 con 0x7fa5c815d160 2023-06-22T17:03:49.340 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5c77fe700 1 --2- 172.21.15.202:0/764382500 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c8149010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0001460 tx=0x7fa5c0001ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:49.340 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5aeffd700 1 -- 172.21.15.202:0/764382500 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000a890 con 0x7fa5c815d160 2023-06-22T17:03:49.340 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5aeffd700 1 -- 172.21.15.202:0/764382500 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c000a9f0 con 0x7fa5c815d160 2023-06-22T17:03:49.340 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5cda5d700 1 -- 172.21.15.202:0/764382500 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c80c8290 con 0x7fa5c815d160 2023-06-22T17:03:49.341 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5aeffd700 1 -- 172.21.15.202:0/764382500 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000ad70 con 0x7fa5c815d160 2023-06-22T17:03:49.341 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5cda5d700 1 -- 172.21.15.202:0/764382500 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c80b3be0 con 0x7fa5c815d160 2023-06-22T17:03:49.341 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5aeffd700 1 -- 172.21.15.202:0/764382500 <== mon.2 v2:172.21.15.202:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa5c0007300 con 0x7fa5c815d160 2023-06-22T17:03:49.341 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.334+0000 7fa5aeffd700 1 -- 172.21.15.202:0/764382500 <== mon.2 v2:172.21.15.202:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fa5c001b850 con 0x7fa5c815d160 2023-06-22T17:03:49.341 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.335+0000 7fa5cda5d700 1 -- 172.21.15.202:0/764382500 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c804c1b0 con 0x7fa5c815d160 2023-06-22T17:03:49.341 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.339+0000 7fa5aeffd700 1 -- 172.21.15.202:0/764382500 <== mon.2 v2:172.21.15.202:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fa5c0007790 con 0x7fa5c815d160 2023-06-22T17:03:49.423 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.422+0000 7fa5cda5d700 1 -- 172.21.15.202:0/764382500 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d426e8cd-b1e6-40a3-9e65-cf52fac95a21", "id": 1} v 0) v1 -- 0x7fa5c80b3e30 con 0x7fa5c815d160 2023-06-22T17:03:49.426 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.425+0000 7fa5aeffd700 1 -- 172.21.15.202:0/764382500 <== mon.2 v2:172.21.15.202:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d426e8cd-b1e6-40a3-9e65-cf52fac95a21", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa5c0007930 con 0x7fa5c815d160 2023-06-22T17:03:49.426 INFO:teuthology.orchestra.run.smithi202.stdout:1 2023-06-22T17:03:49.427 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.425+0000 7fa5cda5d700 1 -- 172.21.15.202:0/764382500 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 msgr2=0x7fa5c8149010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.427 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.425+0000 7fa5cda5d700 1 --2- 172.21.15.202:0/764382500 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c8149010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0001460 tx=0x7fa5c0001ae0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.427 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.426+0000 7fa5cda5d700 1 -- 172.21.15.202:0/764382500 shutdown_connections 2023-06-22T17:03:49.427 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.426+0000 7fa5cda5d700 1 --2- 172.21.15.202:0/764382500 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa5c815d160 0x7fa5c8149010 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.427 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.426+0000 7fa5cda5d700 1 --2- 172.21.15.202:0/764382500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c80d0a40 0x7fa5c80b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.428 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.426+0000 7fa5cda5d700 1 --2- 172.21.15.202:0/764382500 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa5c8156ff0 0x7fa5c80c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.428 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.426+0000 7fa5cda5d700 1 -- 172.21.15.202:0/764382500 >> 172.21.15.202:0/764382500 conn(0x7fa5c8070290 msgr2=0x7fa5c8070660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:49.428 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.426+0000 7fa5cda5d700 1 -- 172.21.15.202:0/764382500 shutdown_connections 2023-06-22T17:03:49.428 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.426+0000 7fa5cda5d700 1 -- 172.21.15.202:0/764382500 wait complete. 2023-06-22T17:03:49.438 DEBUG:teuthology.orchestra.run.smithi202:> sudo ceph --cluster ceph osd new 487a8fca-1da6-4cb3-a73c-abc6e9e2a29f 2 2023-06-22T17:03:49.554 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.553+0000 7fa668e4b700 1 Processor -- start 2023-06-22T17:03:49.555 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.553+0000 7fa668e4b700 1 -- start start 2023-06-22T17:03:49.556 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa668e4b700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fa664153a80 0x7fa6640d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.556 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa668e4b700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa66415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.556 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa668e4b700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6640cedd0 con 0x7fa6640b2010 2023-06-22T17:03:49.557 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa668e4b700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6640d14c0 con 0x7fa664153a80 2023-06-22T17:03:49.557 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa668e4b700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640ceac0 con 0x7fa664153e50 2023-06-22T17:03:49.557 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa661d9b700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa66415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.557 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa66259c700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fa664153a80 0x7fa6640d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.557 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa661d9b700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa66415a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3301/0 says I am v2:172.21.15.202:55102/0 (socket says 172.21.15.202:55102) 2023-06-22T17:03:49.557 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa66259c700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fa664153a80 0x7fa6640d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3300/0 says I am v2:172.21.15.202:46076/0 (socket says 172.21.15.202:46076) 2023-06-22T17:03:49.557 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa662d9d700 1 --1- >> v1:172.21.15.202:6789/0 conn(0x7fa6640b2010 0x7fa66415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.202:6789/0 says I am v1:172.21.15.202:51970/0 (socket says 172.21.15.202:51970) 2023-06-22T17:03:49.557 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.554+0000 7fa661d9b700 1 -- 172.21.15.202:0/26051348 learned_addr learned my addr 172.21.15.202:0/26051348 (peer_addr_for_me v2:172.21.15.202:0/0) 2023-06-22T17:03:49.558 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa66159a700 1 -- 172.21.15.202:0/26051348 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343297096 0 0) 0x7fa6640cedd0 con 0x7fa6640b2010 2023-06-22T17:03:49.558 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa66159a700 1 -- 172.21.15.202:0/26051348 --> v1:172.21.15.202:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa64c003570 con 0x7fa6640b2010 2023-06-22T17:03:49.558 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa661d9b700 1 -- 172.21.15.202:0/26051348 >> v2:172.21.15.202:3300/0 conn(0x7fa664153a80 msgr2=0x7fa6640d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:49.558 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa661d9b700 1 --2- 172.21.15.202:0/26051348 >> v2:172.21.15.202:3300/0 conn(0x7fa664153a80 0x7fa6640d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.558 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa661d9b700 1 -- 172.21.15.202:0/26051348 >> v1:172.21.15.202:6789/0 conn(0x7fa6640b2010 legacy=0x7fa66415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.558 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa661d9b700 1 -- 172.21.15.202:0/26051348 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6640d0c20 con 0x7fa664153e50 2023-06-22T17:03:49.558 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa66259c700 1 --2- 172.21.15.202:0/26051348 >> v2:172.21.15.202:3300/0 conn(0x7fa664153a80 0x7fa6640d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T17:03:49.559 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa661d9b700 1 --2- 172.21.15.202:0/26051348 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa66415a7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa6540099b0 tx=0x7fa654009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ff319436234a380 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:49.559 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa66159a700 1 -- 172.21.15.202:0/26051348 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400b680 con 0x7fa664153e50 2023-06-22T17:03:49.559 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa66159a700 1 -- 172.21.15.202:0/26051348 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65400b7e0 con 0x7fa664153e50 2023-06-22T17:03:49.559 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.555+0000 7fa66159a700 1 -- 172.21.15.202:0/26051348 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400bb60 con 0x7fa664153e50 2023-06-22T17:03:49.561 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.556+0000 7fa668e4b700 1 -- 172.21.15.202:0/26051348 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 msgr2=0x7fa66415a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.561 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.556+0000 7fa668e4b700 1 --2- 172.21.15.202:0/26051348 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa66415a7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa6540099b0 tx=0x7fa654009250 comp rx=0 tx=0).stop 2023-06-22T17:03:49.561 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.556+0000 7fa668e4b700 1 -- 172.21.15.202:0/26051348 shutdown_connections 2023-06-22T17:03:49.561 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.556+0000 7fa668e4b700 1 --2- 172.21.15.202:0/26051348 >> v2:172.21.15.202:3300/0 conn(0x7fa664153a80 0x7fa6640d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.562 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.556+0000 7fa668e4b700 1 --2- 172.21.15.202:0/26051348 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa66415a7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.562 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.556+0000 7fa668e4b700 1 -- 172.21.15.202:0/26051348 >> 172.21.15.202:0/26051348 conn(0x7fa664070290 msgr2=0x7fa664070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:49.562 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.556+0000 7fa668e4b700 1 -- 172.21.15.202:0/26051348 shutdown_connections 2023-06-22T17:03:49.562 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.556+0000 7fa668e4b700 1 -- 172.21.15.202:0/26051348 wait complete. 2023-06-22T17:03:49.562 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.557+0000 7fa668e4b700 1 Processor -- start 2023-06-22T17:03:49.562 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.557+0000 7fa668e4b700 1 -- start start 2023-06-22T17:03:49.562 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa668e4b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6640b2010 0x7fa66414fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.563 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa668e4b700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa664153a80 0x7fa664070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.563 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa668e4b700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa664070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.563 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa668e4b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa664157c00 con 0x7fa6640b2010 2023-06-22T17:03:49.563 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa668e4b700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66415c9b0 con 0x7fa664153a80 2023-06-22T17:03:49.563 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa668e4b700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66416dd10 con 0x7fa664153e50 2023-06-22T17:03:49.563 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa661d9b700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa664153a80 0x7fa664070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.563 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa662d9d700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa664070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.563 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa661d9b700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa664153a80 0x7fa664070660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3300/0 says I am v2:172.21.15.202:46086/0 (socket says 172.21.15.202:46086) 2023-06-22T17:03:49.564 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa661d9b700 1 -- 172.21.15.202:0/2168755325 learned_addr learned my addr 172.21.15.202:0/2168755325 (peer_addr_for_me v2:172.21.15.202:0/0) 2023-06-22T17:03:49.564 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa66259c700 1 --2- 172.21.15.202:0/2168755325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6640b2010 0x7fa66414fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.564 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa661d9b700 1 -- 172.21.15.202:0/2168755325 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 msgr2=0x7fa664070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.564 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa661d9b700 1 --2- 172.21.15.202:0/2168755325 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa664070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.564 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa661d9b700 1 -- 172.21.15.202:0/2168755325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6640b2010 msgr2=0x7fa66414fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.564 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa661d9b700 1 --2- 172.21.15.202:0/2168755325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6640b2010 0x7fa66414fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.564 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa661d9b700 1 -- 172.21.15.202:0/2168755325 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa658009420 con 0x7fa664153a80 2023-06-22T17:03:49.564 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa662d9d700 1 --2- 172.21.15.202:0/2168755325 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa664070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T17:03:49.565 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa661d9b700 1 --2- 172.21.15.202:0/2168755325 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa664153a80 0x7fa664070660 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa654006fd0 tx=0x7fa654017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:49.565 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa6527fc700 1 -- 172.21.15.202:0/2168755325 <== mon.1 v2:172.21.15.202:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400b950 con 0x7fa664153a80 2023-06-22T17:03:49.565 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa6527fc700 1 -- 172.21.15.202:0/2168755325 <== mon.1 v2:172.21.15.202:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65400bad0 con 0x7fa664153a80 2023-06-22T17:03:49.565 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa668e4b700 1 -- 172.21.15.202:0/2168755325 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6540094f0 con 0x7fa664153a80 2023-06-22T17:03:49.565 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa6527fc700 1 -- 172.21.15.202:0/2168755325 <== mon.1 v2:172.21.15.202:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65401a6a0 con 0x7fa664153a80 2023-06-22T17:03:49.565 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa668e4b700 1 -- 172.21.15.202:0/2168755325 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66415c360 con 0x7fa664153a80 2023-06-22T17:03:49.565 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.558+0000 7fa6527fc700 1 -- 172.21.15.202:0/2168755325 <== mon.1 v2:172.21.15.202:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa65401aa10 con 0x7fa664153a80 2023-06-22T17:03:49.566 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.559+0000 7fa6527fc700 1 -- 172.21.15.202:0/2168755325 <== mon.1 v2:172.21.15.202:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fa65402eb90 con 0x7fa664153a80 2023-06-22T17:03:49.566 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.559+0000 7fa668e4b700 1 -- 172.21.15.202:0/2168755325 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66415cb20 con 0x7fa664153a80 2023-06-22T17:03:49.566 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.561+0000 7fa6527fc700 1 -- 172.21.15.202:0/2168755325 <== mon.1 v2:172.21.15.202:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fa654033060 con 0x7fa664153a80 2023-06-22T17:03:49.603 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:49.602+0000 7fbc2e468180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-22T17:03:49.641 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:49.640+0000 7f0769aba180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-22T17:03:49.643 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.642+0000 7fa668e4b700 1 -- 172.21.15.202:0/2168755325 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_command({"prefix": "osd new", "uuid": "487a8fca-1da6-4cb3-a73c-abc6e9e2a29f", "id": 2} v 0) v1 -- 0x7fa66404c1b0 con 0x7fa664153a80 2023-06-22T17:03:49.648 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.645+0000 7fa6527fc700 1 -- 172.21.15.202:0/2168755325 <== mon.1 v2:172.21.15.202:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "487a8fca-1da6-4cb3-a73c-abc6e9e2a29f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa654029070 con 0x7fa664153a80 2023-06-22T17:03:49.648 INFO:teuthology.orchestra.run.smithi202.stdout:2 2023-06-22T17:03:49.649 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.646+0000 7fa668e4b700 1 -- 172.21.15.202:0/2168755325 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa664153a80 msgr2=0x7fa664070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.649 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.646+0000 7fa668e4b700 1 --2- 172.21.15.202:0/2168755325 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa664153a80 0x7fa664070660 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa654006fd0 tx=0x7fa654017000 comp rx=0 tx=0).stop 2023-06-22T17:03:49.650 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.646+0000 7fa668e4b700 1 -- 172.21.15.202:0/2168755325 shutdown_connections 2023-06-22T17:03:49.650 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.646+0000 7fa668e4b700 1 --2- 172.21.15.202:0/2168755325 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa664153e50 0x7fa664070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.650 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.646+0000 7fa668e4b700 1 --2- 172.21.15.202:0/2168755325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6640b2010 0x7fa66414fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.650 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.646+0000 7fa668e4b700 1 --2- 172.21.15.202:0/2168755325 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa664153a80 0x7fa664070660 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.650 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.646+0000 7fa668e4b700 1 -- 172.21.15.202:0/2168755325 >> 172.21.15.202:0/2168755325 conn(0x7fa664070290 msgr2=0x7fa6640b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:49.651 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.646+0000 7fa668e4b700 1 -- 172.21.15.202:0/2168755325 shutdown_connections 2023-06-22T17:03:49.651 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.647+0000 7fa668e4b700 1 -- 172.21.15.202:0/2168755325 wait complete. 2023-06-22T17:03:49.659 DEBUG:teuthology.orchestra.run.smithi202:> sudo ceph --cluster ceph osd new 7089499f-dc69-4d6b-a6c3-5b30404ab1bc 3 2023-06-22T17:03:49.740 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:49.739+0000 7fbc2e468180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-22T17:03:49.776 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.775+0000 7f6383930700 1 Processor -- start 2023-06-22T17:03:49.776 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.775+0000 7f6383930700 1 -- start start 2023-06-22T17:03:49.777 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6383930700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f637c15d160 0x7f637c0ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.778 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6383930700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f637c0cbed0 0x7f637c0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.778 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6383930700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637c0cea30 con 0x7f637c15cd90 2023-06-22T17:03:49.778 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6383930700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c0ced10 con 0x7f637c15d160 2023-06-22T17:03:49.778 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6383930700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c0ceff0 con 0x7f637c0cbed0 2023-06-22T17:03:49.778 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6381ecd700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f637c0cbed0 0x7f637c0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.778 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6381ecd700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f637c0cbed0 0x7f637c0ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3300/0 says I am v2:172.21.15.202:46092/0 (socket says 172.21.15.202:46092) 2023-06-22T17:03:49.778 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6381ecd700 1 -- 172.21.15.202:0/1793698216 learned_addr learned my addr 172.21.15.202:0/1793698216 (peer_addr_for_me v2:172.21.15.202:0/0) 2023-06-22T17:03:49.779 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6380ecb700 1 --2- 172.21.15.202:0/1793698216 >> v2:172.21.15.47:3300/0 conn(0x7f637c15d160 0x7f637c0ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.779 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6381ecd700 1 -- 172.21.15.202:0/1793698216 >> v2:172.21.15.47:3300/0 conn(0x7f637c15d160 msgr2=0x7f637c0ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:49.779 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6381ecd700 1 --2- 172.21.15.202:0/1793698216 >> v2:172.21.15.47:3300/0 conn(0x7f637c15d160 0x7f637c0ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.779 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6381ecd700 1 -- 172.21.15.202:0/1793698216 >> v1:172.21.15.47:6789/0 conn(0x7f637c15cd90 legacy=0x7f637c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.779 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6381ecd700 1 -- 172.21.15.202:0/1793698216 --> v2:172.21.15.202:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0cf2d0 con 0x7f637c0cbed0 2023-06-22T17:03:49.779 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6381ecd700 1 --2- 172.21.15.202:0/1793698216 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0cbed0 0x7f637c0ccf60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f637c0cdf90 tx=0x7f6374009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232ca26d8205a388 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:49.779 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.776+0000 7f6373fff700 1 -- 172.21.15.202:0/1793698216 <== mon.1 v2:172.21.15.202:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637400e070 con 0x7f637c0cbed0 2023-06-22T17:03:49.780 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.777+0000 7f6373fff700 1 -- 172.21.15.202:0/1793698216 <== mon.1 v2:172.21.15.202:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637400ba20 con 0x7f637c0cbed0 2023-06-22T17:03:49.780 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.777+0000 7f6373fff700 1 -- 172.21.15.202:0/1793698216 <== mon.1 v2:172.21.15.202:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63740092e0 con 0x7f637c0cbed0 2023-06-22T17:03:49.782 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.777+0000 7f6383930700 1 -- 172.21.15.202:0/1793698216 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0cbed0 msgr2=0x7f637c0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.782 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.777+0000 7f6383930700 1 --2- 172.21.15.202:0/1793698216 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0cbed0 0x7f637c0ccf60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f637c0cdf90 tx=0x7f6374009840 comp rx=0 tx=0).stop 2023-06-22T17:03:49.782 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.777+0000 7f6383930700 1 -- 172.21.15.202:0/1793698216 shutdown_connections 2023-06-22T17:03:49.782 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.777+0000 7f6383930700 1 --2- 172.21.15.202:0/1793698216 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0cbed0 0x7f637c0ccf60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.782 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.777+0000 7f6383930700 1 --2- 172.21.15.202:0/1793698216 >> v2:172.21.15.47:3300/0 conn(0x7f637c15d160 0x7f637c0ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.783 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.777+0000 7f6383930700 1 -- 172.21.15.202:0/1793698216 >> 172.21.15.202:0/1793698216 conn(0x7f637c099900 msgr2=0x7f637c070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:49.783 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.777+0000 7f6383930700 1 -- 172.21.15.202:0/1793698216 shutdown_connections 2023-06-22T17:03:49.783 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.777+0000 7f6383930700 1 -- 172.21.15.202:0/1793698216 wait complete. 2023-06-22T17:03:49.783 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.778+0000 7f6383930700 1 Processor -- start 2023-06-22T17:03:49.783 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.778+0000 7f6383930700 1 -- start start 2023-06-22T17:03:49.783 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f6383930700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f637c0cbed0 0x7f637c0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.783 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f6383930700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f637c15cd90 0x7f637c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.784 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f6383930700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0c76c0 0x7f637c09daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:49.784 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f6383930700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0c87d0 con 0x7f637c15cd90 2023-06-22T17:03:49.784 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f6383930700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c151280 con 0x7f637c0c76c0 2023-06-22T17:03:49.784 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f6383930700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0c7bc0 con 0x7f637c0cbed0 2023-06-22T17:03:49.784 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f63816cc700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f637c0cbed0 0x7f637c0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.784 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f6381ecd700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0c76c0 0x7f637c09daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.784 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f63816cc700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f637c0cbed0 0x7f637c0c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3301/0 says I am v2:172.21.15.202:55106/0 (socket says 172.21.15.202:55106) 2023-06-22T17:03:49.784 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f6381ecd700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0c76c0 0x7f637c09daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3300/0 says I am v2:172.21.15.202:46100/0 (socket says 172.21.15.202:46100) 2023-06-22T17:03:49.785 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f63816cc700 1 -- 172.21.15.202:0/4212899092 learned_addr learned my addr 172.21.15.202:0/4212899092 (peer_addr_for_me v2:172.21.15.202:0/0) 2023-06-22T17:03:49.785 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.779+0000 7f6380ecb700 1 --2- 172.21.15.202:0/4212899092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f637c15cd90 0x7f637c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:49.785 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63816cc700 1 -- 172.21.15.202:0/4212899092 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0c76c0 msgr2=0x7f637c09daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.785 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63816cc700 1 --2- 172.21.15.202:0/4212899092 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0c76c0 0x7f637c09daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.785 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63816cc700 1 -- 172.21.15.202:0/4212899092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f637c15cd90 msgr2=0x7f637c150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.785 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63816cc700 1 --2- 172.21.15.202:0/4212899092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f637c15cd90 0x7f637c150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.785 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63816cc700 1 -- 172.21.15.202:0/4212899092 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6374009ae0 con 0x7f637c0cbed0 2023-06-22T17:03:49.785 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f6381ecd700 1 --2- 172.21.15.202:0/4212899092 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0c76c0 0x7f637c09daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T17:03:49.786 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63816cc700 1 --2- 172.21.15.202:0/4212899092 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f637c0cbed0 0x7f637c0c8c60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f637800c7c0 tx=0x7f6378007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:49.786 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63717fa700 1 -- 172.21.15.202:0/4212899092 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378011590 con 0x7f637c0cbed0 2023-06-22T17:03:49.786 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63717fa700 1 -- 172.21.15.202:0/4212899092 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63780116f0 con 0x7f637c0cbed0 2023-06-22T17:03:49.786 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f6383930700 1 -- 172.21.15.202:0/4212899092 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c0c7ea0 con 0x7f637c0cbed0 2023-06-22T17:03:49.786 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63717fa700 1 -- 172.21.15.202:0/4212899092 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378011ab0 con 0x7f637c0cbed0 2023-06-22T17:03:49.786 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f6383930700 1 -- 172.21.15.202:0/4212899092 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c0c8460 con 0x7f637c0cbed0 2023-06-22T17:03:49.786 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63717fa700 1 -- 172.21.15.202:0/4212899092 <== mon.2 v2:172.21.15.202:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f637801c6c0 con 0x7f637c0cbed0 2023-06-22T17:03:49.787 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.780+0000 7f63717fa700 1 -- 172.21.15.202:0/4212899092 <== mon.2 v2:172.21.15.202:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f6378012420 con 0x7f637c0cbed0 2023-06-22T17:03:49.787 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.781+0000 7f6383930700 1 -- 172.21.15.202:0/4212899092 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637c04c1b0 con 0x7f637c0cbed0 2023-06-22T17:03:49.787 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:49.780+0000 7f0769aba180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-22T17:03:49.787 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.783+0000 7f63717fa700 1 -- 172.21.15.202:0/4212899092 <== mon.2 v2:172.21.15.202:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f6378017070 con 0x7f637c0cbed0 2023-06-22T17:03:49.820 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:49.818+0000 7fbc2e468180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-22T17:03:49.860 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:49.860+0000 7f0769aba180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-22T17:03:49.869 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.868+0000 7f6383930700 1 -- 172.21.15.202:0/4212899092 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7089499f-dc69-4d6b-a6c3-5b30404ab1bc", "id": 3} v 0) v1 -- 0x7f637c0cf5b0 con 0x7f637c0cbed0 2023-06-22T17:03:49.872 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.871+0000 7f63717fa700 1 -- 172.21.15.202:0/4212899092 <== mon.2 v2:172.21.15.202:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7089499f-dc69-4d6b-a6c3-5b30404ab1bc", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f637801c970 con 0x7f637c0cbed0 2023-06-22T17:03:49.872 INFO:teuthology.orchestra.run.smithi202.stdout:3 2023-06-22T17:03:49.873 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.871+0000 7f6383930700 1 -- 172.21.15.202:0/4212899092 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f637c0cbed0 msgr2=0x7f637c0c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:49.873 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.872+0000 7f6383930700 1 --2- 172.21.15.202:0/4212899092 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f637c0cbed0 0x7f637c0c8c60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f637800c7c0 tx=0x7f6378007f70 comp rx=0 tx=0).stop 2023-06-22T17:03:49.873 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.872+0000 7f6383930700 1 -- 172.21.15.202:0/4212899092 shutdown_connections 2023-06-22T17:03:49.873 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.872+0000 7f6383930700 1 --2- 172.21.15.202:0/4212899092 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f637c0c76c0 0x7f637c09daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.873 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.872+0000 7f6383930700 1 --2- 172.21.15.202:0/4212899092 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f637c0cbed0 0x7f637c0c8c60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.874 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.872+0000 7f6383930700 1 --2- 172.21.15.202:0/4212899092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f637c15cd90 0x7f637c150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:49.874 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.872+0000 7f6383930700 1 -- 172.21.15.202:0/4212899092 >> 172.21.15.202:0/4212899092 conn(0x7f637c099900 msgr2=0x7f637c15bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:49.874 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.872+0000 7f6383930700 1 -- 172.21.15.202:0/4212899092 shutdown_connections 2023-06-22T17:03:49.874 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.872+0000 7f6383930700 1 -- 172.21.15.202:0/4212899092 wait complete. 2023-06-22T17:03:49.885 DEBUG:teuthology.orchestra.run.smithi202:> sudo ceph --cluster ceph osd new f8b34dee-7945-42fd-9e58-98e5fe196b36 4 2023-06-22T17:03:50.000 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.999+0000 7f97bfe32700 1 Processor -- start 2023-06-22T17:03:50.001 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:49.999+0000 7f97bfe32700 1 -- start start 2023-06-22T17:03:50.003 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.000+0000 7f97bfe32700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 0x7f97b80d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.003 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.000+0000 7f97bfe32700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b80cedd0 con 0x7f97b8153e50 2023-06-22T17:03:50.004 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.000+0000 7f97bfe32700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b815aed0 con 0x7f97b80b2010 2023-06-22T17:03:50.004 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.000+0000 7f97bfe32700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80d14c0 con 0x7f97b8153a80 2023-06-22T17:03:50.004 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.000+0000 7f97bdbce700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 0x7f97b80d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.004 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.000+0000 7f97be3cf700 1 --1- >> v1:172.21.15.202:6789/0 conn(0x7f97b80b2010 0x7f97b80cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.202:6789/0 says I am v1:172.21.15.202:51972/0 (socket says 172.21.15.202:51972) 2023-06-22T17:03:50.004 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.000+0000 7f97be3cf700 1 -- 172.21.15.202:0/598639457 learned_addr learned my addr 172.21.15.202:0/598639457 (peer_addr_for_me v1:172.21.15.202:0/0) 2023-06-22T17:03:50.005 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.000+0000 7f97bcbcc700 1 -- 172.21.15.202:0/598639457 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119860300 0 0) 0x7f97b815aed0 con 0x7f97b80b2010 2023-06-22T17:03:50.005 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.001+0000 7f97bcbcc700 1 -- 172.21.15.202:0/598639457 --> v1:172.21.15.202:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a4003570 con 0x7f97b80b2010 2023-06-22T17:03:50.005 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.001+0000 7f97bdbce700 1 -- 172.21.15.202:0/598639457 >> v1:172.21.15.202:6789/0 conn(0x7f97b80b2010 legacy=0x7f97b80cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.005 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.001+0000 7f97bdbce700 1 -- 172.21.15.202:0/598639457 >> v1:172.21.15.47:6789/0 conn(0x7f97b8153e50 legacy=0x7f97b815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.005 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.001+0000 7f97bdbce700 1 -- 172.21.15.202:0/598639457 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80ceac0 con 0x7f97b8153a80 2023-06-22T17:03:50.005 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.001+0000 7f97bdbce700 1 --2- 172.21.15.202:0/598639457 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 0x7f97b80d0590 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009910 tx=0x7f97a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c553dc46f62346 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:50.005 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.001+0000 7f97bcbcc700 1 -- 172.21.15.202:0/598639457 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a800b680 con 0x7f97b8153a80 2023-06-22T17:03:50.006 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.001+0000 7f97bcbcc700 1 -- 172.21.15.202:0/598639457 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a800b7e0 con 0x7f97b8153a80 2023-06-22T17:03:50.006 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.001+0000 7f97bcbcc700 1 -- 172.21.15.202:0/598639457 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a800bba0 con 0x7f97b8153a80 2023-06-22T17:03:50.006 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.002+0000 7f97bfe32700 1 -- 172.21.15.202:0/598639457 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 msgr2=0x7f97b80d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.006 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.002+0000 7f97bfe32700 1 --2- 172.21.15.202:0/598639457 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 0x7f97b80d0590 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009910 tx=0x7f97a80091a0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.006 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.002+0000 7f97bfe32700 1 -- 172.21.15.202:0/598639457 shutdown_connections 2023-06-22T17:03:50.006 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.002+0000 7f97bfe32700 1 --2- 172.21.15.202:0/598639457 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 0x7f97b80d0590 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.006 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.002+0000 7f97bfe32700 1 -- 172.21.15.202:0/598639457 >> 172.21.15.202:0/598639457 conn(0x7f97b8070290 msgr2=0x7f97b8070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:50.007 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.002+0000 7f97bfe32700 1 -- 172.21.15.202:0/598639457 shutdown_connections 2023-06-22T17:03:50.007 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.002+0000 7f97bfe32700 1 -- 172.21.15.202:0/598639457 wait complete. 2023-06-22T17:03:50.009 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.003+0000 7f97bfe32700 1 Processor -- start 2023-06-22T17:03:50.009 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.003+0000 7f97bfe32700 1 -- start start 2023-06-22T17:03:50.009 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.003+0000 7f97bfe32700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97b80b2010 0x7f97b814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.009 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.003+0000 7f97bfe32700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 0x7f97b814b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.010 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.003+0000 7f97bfe32700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f97b8153e50 0x7f97b81507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.010 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.003+0000 7f97bfe32700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b814cd10 con 0x7f97b80b2010 2023-06-22T17:03:50.010 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.003+0000 7f97bfe32700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b814bd10 con 0x7f97b8153e50 2023-06-22T17:03:50.010 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.003+0000 7f97bfe32700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b81510a0 con 0x7f97b8153a80 2023-06-22T17:03:50.011 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97be3cf700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f97b8153e50 0x7f97b81507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.011 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97bd3cd700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 0x7f97b814b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.011 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97be3cf700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f97b8153e50 0x7f97b81507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3300/0 says I am v2:172.21.15.202:46104/0 (socket says 172.21.15.202:46104) 2023-06-22T17:03:50.011 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97be3cf700 1 -- 172.21.15.202:0/2210919064 learned_addr learned my addr 172.21.15.202:0/2210919064 (peer_addr_for_me v2:172.21.15.202:0/0) 2023-06-22T17:03:50.012 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97bdbce700 1 --2- 172.21.15.202:0/2210919064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97b80b2010 0x7f97b814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.012 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97bd3cd700 1 -- 172.21.15.202:0/2210919064 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f97b8153e50 msgr2=0x7f97b81507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.012 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97bd3cd700 1 --2- 172.21.15.202:0/2210919064 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f97b8153e50 0x7f97b81507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.013 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97bd3cd700 1 -- 172.21.15.202:0/2210919064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97b80b2010 msgr2=0x7f97b814c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.013 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97bd3cd700 1 --2- 172.21.15.202:0/2210919064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97b80b2010 0x7f97b814c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.013 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97bd3cd700 1 -- 172.21.15.202:0/2210919064 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8009420 con 0x7f97b8153a80 2023-06-22T17:03:50.013 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97be3cf700 1 --2- 172.21.15.202:0/2210919064 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f97b8153e50 0x7f97b81507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T17:03:50.014 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97bd3cd700 1 --2- 172.21.15.202:0/2210919064 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 0x7f97b814b110 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97b400c590 tx=0x7f97b4007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:50.014 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97adffb700 1 -- 172.21.15.202:0/2210919064 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4011590 con 0x7f97b8153a80 2023-06-22T17:03:50.014 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97adffb700 1 -- 172.21.15.202:0/2210919064 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b4011710 con 0x7f97b8153a80 2023-06-22T17:03:50.014 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97adffb700 1 -- 172.21.15.202:0/2210919064 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4011ad0 con 0x7f97b8153a80 2023-06-22T17:03:50.015 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97bfe32700 1 -- 172.21.15.202:0/2210919064 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b8151380 con 0x7f97b8153a80 2023-06-22T17:03:50.015 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97adffb700 1 -- 172.21.15.202:0/2210919064 <== mon.2 v2:172.21.15.202:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f97b40126a0 con 0x7f97b8153a80 2023-06-22T17:03:50.015 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.004+0000 7f97bfe32700 1 -- 172.21.15.202:0/2210919064 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b814c2b0 con 0x7f97b8153a80 2023-06-22T17:03:50.015 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.005+0000 7f97adffb700 1 -- 172.21.15.202:0/2210919064 <== mon.2 v2:172.21.15.202:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f97b4012b80 con 0x7f97b8153a80 2023-06-22T17:03:50.016 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.005+0000 7f97bfe32700 1 -- 172.21.15.202:0/2210919064 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b804c1b0 con 0x7f97b8153a80 2023-06-22T17:03:50.016 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.007+0000 7f97adffb700 1 -- 172.21.15.202:0/2210919064 <== mon.2 v2:172.21.15.202:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f97b4017070 con 0x7f97b8153a80 2023-06-22T17:03:50.016 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:50.013+0000 7fbc2e468180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-22T17:03:50.054 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:50.053+0000 7f0769aba180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-22T17:03:50.091 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.089+0000 7f97bfe32700 1 -- 172.21.15.202:0/2210919064 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f8b34dee-7945-42fd-9e58-98e5fe196b36", "id": 4} v 0) v1 -- 0x7f97b80d0c20 con 0x7f97b8153a80 2023-06-22T17:03:50.093 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.092+0000 7f97adffb700 1 -- 172.21.15.202:0/2210919064 <== mon.2 v2:172.21.15.202:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f8b34dee-7945-42fd-9e58-98e5fe196b36", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f97b401c440 con 0x7f97b8153a80 2023-06-22T17:03:50.094 INFO:teuthology.orchestra.run.smithi202.stdout:4 2023-06-22T17:03:50.095 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.093+0000 7f97bfe32700 1 -- 172.21.15.202:0/2210919064 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 msgr2=0x7f97b814b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.095 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.093+0000 7f97bfe32700 1 --2- 172.21.15.202:0/2210919064 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 0x7f97b814b110 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97b400c590 tx=0x7f97b4007ec0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.095 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.093+0000 7f97bfe32700 1 -- 172.21.15.202:0/2210919064 shutdown_connections 2023-06-22T17:03:50.095 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.093+0000 7f97bfe32700 1 --2- 172.21.15.202:0/2210919064 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f97b8153e50 0x7f97b81507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.095 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.094+0000 7f97bfe32700 1 --2- 172.21.15.202:0/2210919064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97b80b2010 0x7f97b814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.095 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.094+0000 7f97bfe32700 1 --2- 172.21.15.202:0/2210919064 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f97b8153a80 0x7f97b814b110 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.096 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.094+0000 7f97bfe32700 1 -- 172.21.15.202:0/2210919064 >> 172.21.15.202:0/2210919064 conn(0x7f97b8070290 msgr2=0x7f97b815b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:50.096 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.094+0000 7f97bfe32700 1 -- 172.21.15.202:0/2210919064 shutdown_connections 2023-06-22T17:03:50.096 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.094+0000 7f97bfe32700 1 -- 172.21.15.202:0/2210919064 wait complete. 2023-06-22T17:03:50.106 DEBUG:teuthology.orchestra.run.smithi202:> sudo ceph --cluster ceph osd new 9583b379-49e1-4b6c-adf5-6f19322cc76c 5 2023-06-22T17:03:50.194 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:50.193+0000 7fbc2e468180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-22T17:03:50.221 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.220+0000 7f152759e700 1 Processor -- start 2023-06-22T17:03:50.222 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.220+0000 7f152759e700 1 -- start start 2023-06-22T17:03:50.224 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f152759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1528153900 0x7f15280ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.225 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f152759e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15280b2d80 con 0x7f1528153cd0 2023-06-22T17:03:50.225 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f152759e700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1528159d30 con 0x7f15280cab80 2023-06-22T17:03:50.225 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f152759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1528159a90 con 0x7f1528153900 2023-06-22T17:03:50.225 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f1526d9d700 1 --1- >> v1:172.21.15.202:6789/0 conn(0x7f15280cab80 0x7f152815a1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.202:6789/0 says I am v1:172.21.15.202:51984/0 (socket says 172.21.15.202:51984) 2023-06-22T17:03:50.225 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f1526d9d700 1 -- 172.21.15.202:0/762984585 learned_addr learned my addr 172.21.15.202:0/762984585 (peer_addr_for_me v1:172.21.15.202:0/0) 2023-06-22T17:03:50.225 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f152659c700 1 --2- 172.21.15.202:0/762984585 >> v2:172.21.15.47:3300/0 conn(0x7f1528153900 0x7f15280ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.226 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262336446 0 0) 0x7f1528159d30 con 0x7f15280cab80 2023-06-22T17:03:50.226 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 --> v1:172.21.15.202:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1510003570 con 0x7f15280cab80 2023-06-22T17:03:50.226 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 961197406 0 0) 0x7f1518001110 con 0x7f1528153cd0 2023-06-22T17:03:50.226 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1528159d30 con 0x7f1528153cd0 2023-06-22T17:03:50.226 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.221+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 <== mon.1 v1:172.21.15.202:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1934586006 0 0) 0x7f1510003570 con 0x7f15280cab80 2023-06-22T17:03:50.226 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 >> v2:172.21.15.47:3300/0 conn(0x7f1528153900 msgr2=0x7f15280ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:50.226 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 --2- 172.21.15.202:0/762984585 >> v2:172.21.15.47:3300/0 conn(0x7f1528153900 0x7f15280ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.226 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 >> v1:172.21.15.47:6789/0 conn(0x7f1528153cd0 legacy=0x7f152815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.227 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 --> v1:172.21.15.202:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15280cb000 con 0x7f15280cab80 2023-06-22T17:03:50.227 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 <== mon.1 v1:172.21.15.202:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 763095103 0 0) 0x7f15200017f0 con 0x7f15280cab80 2023-06-22T17:03:50.227 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 >> v1:172.21.15.202:6789/0 conn(0x7f15280cab80 legacy=0x7f152815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.227 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 0x7f1510003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.227 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1510004810 0x7f1510004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.227 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f15100051c0 0x7f151000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.227 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1528159a90 con 0x7f1510004810 2023-06-22T17:03:50.227 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151000c160 con 0x7f1510003bf0 2023-06-22T17:03:50.228 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151000c320 con 0x7f15100051c0 2023-06-22T17:03:50.228 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152659c700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 0x7f1510003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.228 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f1526d9d700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f15100051c0 0x7f151000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.228 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152659c700 1 -- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f15100051c0 msgr2=0x7f151000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:50.228 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152659c700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f15100051c0 0x7f151000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.228 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152659c700 1 -- 172.21.15.202:0/762984585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1510004810 msgr2=0x7f1510004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:50.228 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152659c700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1510004810 0x7f1510004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.229 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152659c700 1 -- 172.21.15.202:0/762984585 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1510003570 con 0x7f1510003bf0 2023-06-22T17:03:50.229 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f1526d9d700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f15100051c0 0x7f151000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T17:03:50.229 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f1526d9d700 1 -- 172.21.15.202:0/762984585 reap_dead start 2023-06-22T17:03:50.229 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152659c700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 0x7f1510003fe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f151c006f00 tx=0x7f151c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e84f4103b5413c3 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:50.229 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 <== mon.1 v2:172.21.15.202:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c00e710 con 0x7f1510003bf0 2023-06-22T17:03:50.229 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 <== mon.1 v2:172.21.15.202:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151c00e870 con 0x7f1510003bf0 2023-06-22T17:03:50.229 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.222+0000 7f152559a700 1 -- 172.21.15.202:0/762984585 <== mon.1 v2:172.21.15.202:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c00ec30 con 0x7f1510003bf0 2023-06-22T17:03:50.229 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.223+0000 7f152759e700 1 -- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 msgr2=0x7f1510003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.230 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.223+0000 7f152759e700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 0x7f1510003fe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f151c006f00 tx=0x7f151c0091a0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.230 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.223+0000 7f152759e700 1 -- 172.21.15.202:0/762984585 shutdown_connections 2023-06-22T17:03:50.230 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.223+0000 7f152759e700 1 --2- 172.21.15.202:0/762984585 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 0x7f1510003fe0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.230 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.223+0000 7f152759e700 1 -- 172.21.15.202:0/762984585 >> 172.21.15.202:0/762984585 conn(0x7f1528070290 msgr2=0x7f1528070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:50.230 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.223+0000 7f152759e700 1 -- 172.21.15.202:0/762984585 shutdown_connections 2023-06-22T17:03:50.230 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.223+0000 7f152759e700 1 -- 172.21.15.202:0/762984585 wait complete. 2023-06-22T17:03:50.232 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.224+0000 7f152759e700 1 Processor -- start 2023-06-22T17:03:50.232 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.224+0000 7f152759e700 1 -- start start 2023-06-22T17:03:50.232 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f152759e700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 0x7f15280b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.233 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f152759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15280b3e10 0x7f15280b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.233 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f152759e700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f1528156640 0x7f1528156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.233 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f152759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280b47c0 con 0x7f15280b3e10 2023-06-22T17:03:50.233 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f152759e700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1528152870 con 0x7f1510003bf0 2023-06-22T17:03:50.233 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f152759e700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1528154d30 con 0x7f1528156640 2023-06-22T17:03:50.233 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f152659c700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 0x7f15280b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.233 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f1526d9d700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f1528156640 0x7f1528156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.234 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f1526d9d700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f1528156640 0x7f1528156a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3301/0 says I am v2:172.21.15.202:55124/0 (socket says 172.21.15.202:55124) 2023-06-22T17:03:50.234 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f1526d9d700 1 -- 172.21.15.202:0/3056220119 learned_addr learned my addr 172.21.15.202:0/3056220119 (peer_addr_for_me v2:172.21.15.202:0/0) 2023-06-22T17:03:50.234 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f1525d9b700 1 --2- 172.21.15.202:0/3056220119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15280b3e10 0x7f15280b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.234 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f1526d9d700 1 -- 172.21.15.202:0/3056220119 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 msgr2=0x7f15280b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.234 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f1526d9d700 1 --2- 172.21.15.202:0/3056220119 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 0x7f15280b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.234 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f1526d9d700 1 -- 172.21.15.202:0/3056220119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15280b3e10 msgr2=0x7f15280b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.234 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f1526d9d700 1 --2- 172.21.15.202:0/3056220119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15280b3e10 0x7f15280b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.234 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f1526d9d700 1 -- 172.21.15.202:0/3056220119 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15280b55e0 con 0x7f1528156640 2023-06-22T17:03:50.235 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f1526d9d700 1 --2- 172.21.15.202:0/3056220119 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f1528156640 0x7f1528156a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1528064370 tx=0x7f15280cc660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:50.235 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.225+0000 7f15167fc700 1 -- 172.21.15.202:0/3056220119 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520007ae0 con 0x7f1528156640 2023-06-22T17:03:50.235 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.226+0000 7f15167fc700 1 -- 172.21.15.202:0/3056220119 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15200133f0 con 0x7f1528156640 2023-06-22T17:03:50.235 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.226+0000 7f152759e700 1 -- 172.21.15.202:0/3056220119 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15280b49d0 con 0x7f1528156640 2023-06-22T17:03:50.235 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.226+0000 7f15167fc700 1 -- 172.21.15.202:0/3056220119 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15200137b0 con 0x7f1528156640 2023-06-22T17:03:50.235 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.226+0000 7f152759e700 1 -- 172.21.15.202:0/3056220119 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15280b5010 con 0x7f1528156640 2023-06-22T17:03:50.235 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.226+0000 7f15167fc700 1 -- 172.21.15.202:0/3056220119 <== mon.2 v2:172.21.15.202:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1520013b20 con 0x7f1528156640 2023-06-22T17:03:50.235 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.226+0000 7f15167fc700 1 -- 172.21.15.202:0/3056220119 <== mon.2 v2:172.21.15.202:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f152001ec10 con 0x7f1528156640 2023-06-22T17:03:50.236 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.227+0000 7f152759e700 1 -- 172.21.15.202:0/3056220119 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15280b5770 con 0x7f1528156640 2023-06-22T17:03:50.236 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.229+0000 7f15167fc700 1 -- 172.21.15.202:0/3056220119 <== mon.2 v2:172.21.15.202:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f152001e690 con 0x7f1528156640 2023-06-22T17:03:50.236 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:50.234+0000 7f0769aba180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-22T17:03:50.275 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:50.274+0000 7fbc2e468180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-22T17:03:50.310 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.309+0000 7f152759e700 1 -- 172.21.15.202:0/3056220119 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9583b379-49e1-4b6c-adf5-6f19322cc76c", "id": 5} v 0) v1 -- 0x7f1528088e00 con 0x7f1528156640 2023-06-22T17:03:50.316 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.314+0000 7f15167fc700 1 -- 172.21.15.202:0/3056220119 <== mon.2 v2:172.21.15.202:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9583b379-49e1-4b6c-adf5-6f19322cc76c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f152001e830 con 0x7f1528156640 2023-06-22T17:03:50.316 INFO:teuthology.orchestra.run.smithi202.stdout:5 2023-06-22T17:03:50.318 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.315+0000 7f152759e700 1 -- 172.21.15.202:0/3056220119 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f1528156640 msgr2=0x7f1528156a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.318 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.315+0000 7f152759e700 1 --2- 172.21.15.202:0/3056220119 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f1528156640 0x7f1528156a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1528064370 tx=0x7f15280cc660 comp rx=0 tx=0).stop 2023-06-22T17:03:50.318 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.316+0000 7f152759e700 1 -- 172.21.15.202:0/3056220119 shutdown_connections 2023-06-22T17:03:50.319 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.316+0000 7f152759e700 1 --2- 172.21.15.202:0/3056220119 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f1528156640 0x7f1528156a90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.319 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.316+0000 7f152759e700 1 --2- 172.21.15.202:0/3056220119 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1510003bf0 0x7f15280b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.319 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.316+0000 7f152759e700 1 --2- 172.21.15.202:0/3056220119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15280b3e10 0x7f15280b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.319 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.316+0000 7f152759e700 1 -- 172.21.15.202:0/3056220119 >> 172.21.15.202:0/3056220119 conn(0x7f1528070290 msgr2=0x7f1528152f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:50.320 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.316+0000 7f152759e700 1 -- 172.21.15.202:0/3056220119 shutdown_connections 2023-06-22T17:03:50.320 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.316+0000 7f152759e700 1 -- 172.21.15.202:0/3056220119 wait complete. 2023-06-22T17:03:50.321 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:50.316+0000 7f0769aba180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-22T17:03:50.330 DEBUG:teuthology.orchestra.run.smithi202:> sudo ceph --cluster ceph osd new d11633b5-5a36-441b-93c7-7c0b434f0dbc 6 2023-06-22T17:03:50.417 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:50.416+0000 7fbc2e468180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-22T17:03:50.447 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.446+0000 7f9b22b77700 1 Processor -- start 2023-06-22T17:03:50.447 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.446+0000 7f9b22b77700 1 -- start start 2023-06-22T17:03:50.449 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b22b77700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153a80 0x7f9b1c0d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.450 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b22b77700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b1c0b2010 0x7f9b1c0b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.450 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b22b77700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1c0cedd0 con 0x7f9b1c153e50 2023-06-22T17:03:50.450 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b22b77700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c0d14c0 con 0x7f9b1c0b2010 2023-06-22T17:03:50.450 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b22b77700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c0b29d0 con 0x7f9b1c153a80 2023-06-22T17:03:50.450 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b20913700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153a80 0x7f9b1c0d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.450 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b1bfff700 1 --1- >> v1:172.21.15.202:6789/0 conn(0x7f9b1c153e50 0x7f9b1c15a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.202:6789/0 says I am v1:172.21.15.202:51996/0 (socket says 172.21.15.202:51996) 2023-06-22T17:03:50.451 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b20913700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153a80 0x7f9b1c0d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3301/0 says I am v2:172.21.15.202:55132/0 (socket says 172.21.15.202:55132) 2023-06-22T17:03:50.451 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b1bfff700 1 -- 172.21.15.202:0/2851830195 learned_addr learned my addr 172.21.15.202:0/2851830195 (peer_addr_for_me v1:172.21.15.202:0/0) 2023-06-22T17:03:50.451 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b21114700 1 --2- 172.21.15.202:0/2851830195 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c0b2010 0x7f9b1c0b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.451 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b20913700 1 -- 172.21.15.202:0/2851830195 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c0b2010 msgr2=0x7f9b1c0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:50.451 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b20913700 1 --2- 172.21.15.202:0/2851830195 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c0b2010 0x7f9b1c0b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.451 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b20913700 1 -- 172.21.15.202:0/2851830195 >> v1:172.21.15.202:6789/0 conn(0x7f9b1c153e50 legacy=0x7f9b1c15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.452 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b1b7fe700 1 -- 172.21.15.202:0/2851830195 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558487794 0 0) 0x7f9b1c0cedd0 con 0x7f9b1c153e50 2023-06-22T17:03:50.452 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b20913700 1 -- 172.21.15.202:0/2851830195 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c15aed0 con 0x7f9b1c153a80 2023-06-22T17:03:50.452 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b20913700 1 --2- 172.21.15.202:0/2851830195 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153a80 0x7f9b1c0d0590 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9b100098e0 tx=0x7f9b100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f564b2187641d26e server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:50.452 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b1b7fe700 1 -- 172.21.15.202:0/2851830195 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1000b640 con 0x7f9b1c153a80 2023-06-22T17:03:50.452 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b1b7fe700 1 -- 172.21.15.202:0/2851830195 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1000b7a0 con 0x7f9b1c153a80 2023-06-22T17:03:50.453 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.447+0000 7f9b1b7fe700 1 -- 172.21.15.202:0/2851830195 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1000bb40 con 0x7f9b1c153a80 2023-06-22T17:03:50.453 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.448+0000 7f9b22b77700 1 -- 172.21.15.202:0/2851830195 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153a80 msgr2=0x7f9b1c0d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.453 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.448+0000 7f9b22b77700 1 --2- 172.21.15.202:0/2851830195 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153a80 0x7f9b1c0d0590 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9b100098e0 tx=0x7f9b100091a0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.453 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.448+0000 7f9b22b77700 1 -- 172.21.15.202:0/2851830195 shutdown_connections 2023-06-22T17:03:50.453 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.448+0000 7f9b22b77700 1 --2- 172.21.15.202:0/2851830195 >> v2:172.21.15.47:3300/0 conn(0x7f9b1c0b2010 0x7f9b1c0b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.453 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.448+0000 7f9b22b77700 1 --2- 172.21.15.202:0/2851830195 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153a80 0x7f9b1c0d0590 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.454 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.448+0000 7f9b22b77700 1 -- 172.21.15.202:0/2851830195 >> 172.21.15.202:0/2851830195 conn(0x7f9b1c070290 msgr2=0x7f9b1c070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:50.454 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.448+0000 7f9b22b77700 1 -- 172.21.15.202:0/2851830195 shutdown_connections 2023-06-22T17:03:50.454 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.448+0000 7f9b22b77700 1 -- 172.21.15.202:0/2851830195 wait complete. 2023-06-22T17:03:50.456 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.449+0000 7f9b22b77700 1 Processor -- start 2023-06-22T17:03:50.456 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.449+0000 7f9b22b77700 1 -- start start 2023-06-22T17:03:50.456 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b22b77700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f9b1c0b2010 0x7f9b1c16ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.457 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b22b77700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b1c153a80 0x7f9b1c16ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.457 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b22b77700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153e50 0x7f9b1c152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.457 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b22b77700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c156380 con 0x7f9b1c153a80 2023-06-22T17:03:50.457 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b22b77700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c16a8a0 con 0x7f9b1c0b2010 2023-06-22T17:03:50.457 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b22b77700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1565d0 con 0x7f9b1c153e50 2023-06-22T17:03:50.458 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b21114700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153e50 0x7f9b1c152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.458 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b21114700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153e50 0x7f9b1c152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3301/0 says I am v2:172.21.15.202:55142/0 (socket says 172.21.15.202:55142) 2023-06-22T17:03:50.458 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b20913700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f9b1c0b2010 0x7f9b1c16ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.458 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b21114700 1 -- 172.21.15.202:0/3779624749 learned_addr learned my addr 172.21.15.202:0/3779624749 (peer_addr_for_me v2:172.21.15.202:0/0) 2023-06-22T17:03:50.458 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b1bfff700 1 --2- 172.21.15.202:0/3779624749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b1c153a80 0x7f9b1c16ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.459 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b21114700 1 -- 172.21.15.202:0/3779624749 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f9b1c0b2010 msgr2=0x7f9b1c16ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.459 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b21114700 1 --2- 172.21.15.202:0/3779624749 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f9b1c0b2010 0x7f9b1c16ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.459 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b21114700 1 -- 172.21.15.202:0/3779624749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b1c153a80 msgr2=0x7f9b1c16ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.459 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b21114700 1 --2- 172.21.15.202:0/3779624749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b1c153a80 0x7f9b1c16ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.459 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.450+0000 7f9b21114700 1 -- 172.21.15.202:0/3779624749 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b10009420 con 0x7f9b1c153e50 2023-06-22T17:03:50.459 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.451+0000 7f9b21114700 1 --2- 172.21.15.202:0/3779624749 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153e50 0x7f9b1c152ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009df0 tx=0x7f9b1400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:50.459 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.451+0000 7f9b18ff9700 1 -- 172.21.15.202:0/3779624749 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1400caa0 con 0x7f9b1c153e50 2023-06-22T17:03:50.460 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.451+0000 7f9b18ff9700 1 -- 172.21.15.202:0/3779624749 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1400f040 con 0x7f9b1c153e50 2023-06-22T17:03:50.460 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.451+0000 7f9b18ff9700 1 -- 172.21.15.202:0/3779624749 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1400ce70 con 0x7f9b1c153e50 2023-06-22T17:03:50.460 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.451+0000 7f9b22b77700 1 -- 172.21.15.202:0/3779624749 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c1568b0 con 0x7f9b1c153e50 2023-06-22T17:03:50.460 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.451+0000 7f9b22b77700 1 -- 172.21.15.202:0/3779624749 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c1531e0 con 0x7f9b1c153e50 2023-06-22T17:03:50.460 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.451+0000 7f9b18ff9700 1 -- 172.21.15.202:0/3779624749 <== mon.2 v2:172.21.15.202:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9b14014690 con 0x7f9b1c153e50 2023-06-22T17:03:50.460 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.451+0000 7f9b18ff9700 1 -- 172.21.15.202:0/3779624749 <== mon.2 v2:172.21.15.202:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f9b1401daa0 con 0x7f9b1c153e50 2023-06-22T17:03:50.460 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.452+0000 7f9b22b77700 1 -- 172.21.15.202:0/3779624749 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1c04c1b0 con 0x7f9b1c153e50 2023-06-22T17:03:50.461 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.454+0000 7f9b18ff9700 1 -- 172.21.15.202:0/3779624749 <== mon.2 v2:172.21.15.202:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f9b1403b6e0 con 0x7f9b1c153e50 2023-06-22T17:03:50.461 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:50.458+0000 7f0769aba180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-22T17:03:50.517 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:50.516+0000 7fbc2e468180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-22T17:03:50.539 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.538+0000 7f9b22b77700 1 -- 172.21.15.202:0/3779624749 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d11633b5-5a36-441b-93c7-7c0b434f0dbc", "id": 6} v 0) v1 -- 0x7f9b1c156dd0 con 0x7f9b1c153e50 2023-06-22T17:03:50.545 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.544+0000 7f9b18ff9700 1 -- 172.21.15.202:0/3779624749 <== mon.2 v2:172.21.15.202:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d11633b5-5a36-441b-93c7-7c0b434f0dbc", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9b1c156dd0 con 0x7f9b1c153e50 2023-06-22T17:03:50.545 INFO:teuthology.orchestra.run.smithi202.stdout:6 2023-06-22T17:03:50.546 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.545+0000 7f9b22b77700 1 -- 172.21.15.202:0/3779624749 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153e50 msgr2=0x7f9b1c152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.546 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.545+0000 7f9b22b77700 1 --2- 172.21.15.202:0/3779624749 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153e50 0x7f9b1c152ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009df0 tx=0x7f9b1400b040 comp rx=0 tx=0).stop 2023-06-22T17:03:50.547 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.545+0000 7f9b22b77700 1 -- 172.21.15.202:0/3779624749 shutdown_connections 2023-06-22T17:03:50.547 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.545+0000 7f9b22b77700 1 --2- 172.21.15.202:0/3779624749 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f9b1c153e50 0x7f9b1c152ca0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.547 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.545+0000 7f9b22b77700 1 --2- 172.21.15.202:0/3779624749 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f9b1c0b2010 0x7f9b1c16ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.547 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.545+0000 7f9b22b77700 1 --2- 172.21.15.202:0/3779624749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b1c153a80 0x7f9b1c16ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.547 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.545+0000 7f9b22b77700 1 -- 172.21.15.202:0/3779624749 >> 172.21.15.202:0/3779624749 conn(0x7f9b1c070290 msgr2=0x7f9b1c15c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:50.547 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.545+0000 7f9b22b77700 1 -- 172.21.15.202:0/3779624749 shutdown_connections 2023-06-22T17:03:50.547 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.545+0000 7f9b22b77700 1 -- 172.21.15.202:0/3779624749 wait complete. 2023-06-22T17:03:50.558 DEBUG:teuthology.orchestra.run.smithi202:> sudo ceph --cluster ceph osd new edd9bf42-9566-4d07-902b-08cf16cb42a4 7 2023-06-22T17:03:50.559 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:50.558+0000 7f0769aba180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-22T17:03:50.673 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.672+0000 7f7f823de700 1 Processor -- start 2023-06-22T17:03:50.674 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.672+0000 7f7f823de700 1 -- start start 2023-06-22T17:03:50.676 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.673+0000 7f7f823de700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7f7c0cf0e0 0x7f7f7c159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.676 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.673+0000 7f7f823de700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0cd0f0 0x7f7f7c0c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.676 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.673+0000 7f7f823de700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c068520 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.676 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.673+0000 7f7f823de700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7c06a540 con 0x7f7f7c0cf0e0 2023-06-22T17:03:50.676 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.673+0000 7f7f823de700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c068770 con 0x7f7f7c0cd0f0 2023-06-22T17:03:50.676 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.673+0000 7f7f8097b700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0cd0f0 0x7f7f7c0c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.677 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.673+0000 7f7f7bfff700 1 --1- >> v1:172.21.15.202:6789/0 conn(0x7f7f7c0ccd20 0x7f7f7c1595b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.202:6789/0 says I am v1:172.21.15.202:52006/0 (socket says 172.21.15.202:52006) 2023-06-22T17:03:50.677 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.673+0000 7f7f7bfff700 1 -- 172.21.15.202:0/80717237 learned_addr learned my addr 172.21.15.202:0/80717237 (peer_addr_for_me v1:172.21.15.202:0/0) 2023-06-22T17:03:50.677 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.673+0000 7f7f7b7fe700 1 --2- 172.21.15.202:0/80717237 >> v2:172.21.15.47:3300/0 conn(0x7f7f7c0cf0e0 0x7f7f7c159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.677 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f7affd700 1 -- 172.21.15.202:0/80717237 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952623370 0 0) 0x7f7f7c068520 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.677 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f8097b700 1 -- 172.21.15.202:0/80717237 >> v2:172.21.15.47:3300/0 conn(0x7f7f7c0cf0e0 msgr2=0x7f7f7c159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:50.677 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f8097b700 1 --2- 172.21.15.202:0/80717237 >> v2:172.21.15.47:3300/0 conn(0x7f7f7c0cf0e0 0x7f7f7c159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.677 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f8097b700 1 -- 172.21.15.202:0/80717237 >> v1:172.21.15.202:6789/0 conn(0x7f7f7c0ccd20 legacy=0x7f7f7c1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.678 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f8097b700 1 -- 172.21.15.202:0/80717237 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c068a50 con 0x7f7f7c0cd0f0 2023-06-22T17:03:50.678 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f8097b700 1 --2- 172.21.15.202:0/80717237 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0cd0f0 0x7f7f7c0c3250 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009e60 tx=0x7f7f74009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74ae9ce494c4077b server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:50.678 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f7affd700 1 -- 172.21.15.202:0/80717237 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400e070 con 0x7f7f7c0cd0f0 2023-06-22T17:03:50.678 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f7affd700 1 -- 172.21.15.202:0/80717237 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7400bc30 con 0x7f7f7c0cd0f0 2023-06-22T17:03:50.678 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f7affd700 1 -- 172.21.15.202:0/80717237 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74012430 con 0x7f7f7c0cd0f0 2023-06-22T17:03:50.678 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f823de700 1 -- 172.21.15.202:0/80717237 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0cd0f0 msgr2=0x7f7f7c0c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.678 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f823de700 1 --2- 172.21.15.202:0/80717237 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0cd0f0 0x7f7f7c0c3250 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009e60 tx=0x7f7f74009780 comp rx=0 tx=0).stop 2023-06-22T17:03:50.678 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f823de700 1 -- 172.21.15.202:0/80717237 shutdown_connections 2023-06-22T17:03:50.679 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f823de700 1 --2- 172.21.15.202:0/80717237 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0cd0f0 0x7f7f7c0c3250 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.679 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f823de700 1 --2- 172.21.15.202:0/80717237 >> v2:172.21.15.47:3300/0 conn(0x7f7f7c0cf0e0 0x7f7f7c159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.679 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f823de700 1 -- 172.21.15.202:0/80717237 >> 172.21.15.202:0/80717237 conn(0x7f7f7c067bf0 msgr2=0x7f7f7c0ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:50.679 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f823de700 1 -- 172.21.15.202:0/80717237 shutdown_connections 2023-06-22T17:03:50.679 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.674+0000 7f7f823de700 1 -- 172.21.15.202:0/80717237 wait complete. 2023-06-22T17:03:50.681 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.675+0000 7f7f823de700 1 Processor -- start 2023-06-22T17:03:50.681 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f823de700 1 -- start start 2023-06-22T17:03:50.682 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f823de700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0ccd20 0x7f7f7c154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.682 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f823de700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c0cd0f0 0x7f7f7c15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.682 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f823de700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7f7c0cf0e0 0x7f7f7c15bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.682 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f823de700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0b3dc0 con 0x7f7f7c0cd0f0 2023-06-22T17:03:50.682 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f823de700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c098db0 con 0x7f7f7c0cf0e0 2023-06-22T17:03:50.682 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f823de700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c099070 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.682 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0ccd20 0x7f7f7c154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.683 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f8097b700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7f7c0cf0e0 0x7f7f7c15bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.683 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0ccd20 0x7f7f7c154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3301/0 says I am v2:172.21.15.202:55156/0 (socket says 172.21.15.202:55156) 2023-06-22T17:03:50.683 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.676+0000 7f7f7bfff700 1 -- 172.21.15.202:0/2345162208 learned_addr learned my addr 172.21.15.202:0/2345162208 (peer_addr_for_me v2:172.21.15.202:0/0) 2023-06-22T17:03:50.683 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f7b7fe700 1 --2- 172.21.15.202:0/2345162208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c0cd0f0 0x7f7f7c15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.683 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f7bfff700 1 -- 172.21.15.202:0/2345162208 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7f7c0cf0e0 msgr2=0x7f7f7c15bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.683 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f7bfff700 1 --2- 172.21.15.202:0/2345162208 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7f7c0cf0e0 0x7f7f7c15bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.683 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f7bfff700 1 -- 172.21.15.202:0/2345162208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c0cd0f0 msgr2=0x7f7f7c15b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.683 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f7bfff700 1 --2- 172.21.15.202:0/2345162208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c0cd0f0 0x7f7f7c15b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f7bfff700 1 -- 172.21.15.202:0/2345162208 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74009a20 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f8097b700 1 --2- 172.21.15.202:0/2345162208 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7f7c0cf0e0 0x7f7f7c15bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T17:03:50.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f7bfff700 1 --2- 172.21.15.202:0/2345162208 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0ccd20 0x7f7f7c154500 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00c810 tx=0x7f7f6c007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:50.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f63fff700 1 -- 172.21.15.202:0/2345162208 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0072e0 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f63fff700 1 -- 172.21.15.202:0/2345162208 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c007460 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f823de700 1 -- 172.21.15.202:0/2345162208 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c15c490 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.684 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f63fff700 1 -- 172.21.15.202:0/2345162208 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0057f0 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.685 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f823de700 1 -- 172.21.15.202:0/2345162208 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c15ca50 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.685 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f63fff700 1 -- 172.21.15.202:0/2345162208 <== mon.2 v2:172.21.15.202:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7f6c013400 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.685 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.677+0000 7f7f63fff700 1 -- 172.21.15.202:0/2345162208 <== mon.2 v2:172.21.15.202:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f7f6c001e40 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.685 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.678+0000 7f7f823de700 1 -- 172.21.15.202:0/2345162208 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c0cbc90 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.685 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.680+0000 7f7f63fff700 1 -- 172.21.15.202:0/2345162208 <== mon.2 v2:172.21.15.202:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f7f6c021060 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.760 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.759+0000 7f7f823de700 1 -- 172.21.15.202:0/2345162208 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_command({"prefix": "osd new", "uuid": "edd9bf42-9566-4d07-902b-08cf16cb42a4", "id": 7} v 0) v1 -- 0x7f7f7c04c1b0 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.765 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.764+0000 7f7f63fff700 1 -- 172.21.15.202:0/2345162208 <== mon.2 v2:172.21.15.202:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "edd9bf42-9566-4d07-902b-08cf16cb42a4", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f6c005de0 con 0x7f7f7c0ccd20 2023-06-22T17:03:50.766 INFO:teuthology.orchestra.run.smithi202.stdout:7 2023-06-22T17:03:50.767 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.765+0000 7f7f823de700 1 -- 172.21.15.202:0/2345162208 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0ccd20 msgr2=0x7f7f7c154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.767 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.765+0000 7f7f823de700 1 --2- 172.21.15.202:0/2345162208 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0ccd20 0x7f7f7c154500 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00c810 tx=0x7f7f6c007c80 comp rx=0 tx=0).stop 2023-06-22T17:03:50.767 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.765+0000 7f7f823de700 1 -- 172.21.15.202:0/2345162208 shutdown_connections 2023-06-22T17:03:50.767 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.765+0000 7f7f823de700 1 --2- 172.21.15.202:0/2345162208 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7f7c0cf0e0 0x7f7f7c15bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.767 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.765+0000 7f7f823de700 1 --2- 172.21.15.202:0/2345162208 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7f7c0ccd20 0x7f7f7c154500 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.767 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.765+0000 7f7f823de700 1 --2- 172.21.15.202:0/2345162208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f7c0cd0f0 0x7f7f7c15b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.768 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.765+0000 7f7f823de700 1 -- 172.21.15.202:0/2345162208 >> 172.21.15.202:0/2345162208 conn(0x7f7f7c067bf0 msgr2=0x7f7f7c155200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:50.768 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.765+0000 7f7f823de700 1 -- 172.21.15.202:0/2345162208 shutdown_connections 2023-06-22T17:03:50.768 INFO:teuthology.orchestra.run.smithi202.stderr:2023-06-22T17:03:50.765+0000 7f7f823de700 1 -- 172.21.15.202:0/2345162208 wait complete. 2023-06-22T17:03:50.778 INFO:tasks.ceph.osd.0:Restarting daemon 2023-06-22T17:03:50.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-06-22T17:03:50.780 INFO:tasks.ceph.osd.0:Started 2023-06-22T17:03:50.781 INFO:tasks.ceph.osd.1:Restarting daemon 2023-06-22T17:03:50.781 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-06-22T17:03:50.783 INFO:tasks.ceph.osd.1:Started 2023-06-22T17:03:50.783 INFO:tasks.ceph.osd.2:Restarting daemon 2023-06-22T17:03:50.784 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-06-22T17:03:50.786 INFO:tasks.ceph.osd.2:Started 2023-06-22T17:03:50.787 INFO:tasks.ceph.osd.3:Restarting daemon 2023-06-22T17:03:50.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-06-22T17:03:50.789 INFO:tasks.ceph.osd.3:Started 2023-06-22T17:03:50.789 INFO:tasks.ceph.osd.4:Restarting daemon 2023-06-22T17:03:50.789 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-06-22T17:03:50.791 INFO:tasks.ceph.osd.4:Started 2023-06-22T17:03:50.792 INFO:tasks.ceph.osd.5:Restarting daemon 2023-06-22T17:03:50.792 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-06-22T17:03:50.794 INFO:tasks.ceph.osd.5:Started 2023-06-22T17:03:50.794 INFO:tasks.ceph.osd.6:Restarting daemon 2023-06-22T17:03:50.795 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-06-22T17:03:50.797 INFO:tasks.ceph.osd.6:Started 2023-06-22T17:03:50.798 INFO:tasks.ceph.osd.7:Restarting daemon 2023-06-22T17:03:50.799 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-06-22T17:03:50.801 INFO:tasks.ceph.osd.7:Started 2023-06-22T17:03:50.802 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-22T17:03:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.947+0000 7fe798c65700 1 Processor -- start 2023-06-22T17:03:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.947+0000 7fe798c65700 1 -- start start 2023-06-22T17:03:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe798c65700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fe7940b24e0 0x7fe7940b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe798c65700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7940b52f0 0x7fe7940cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe798c65700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7940cec20 con 0x7fe794142de0 2023-06-22T17:03:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe798c65700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7940b5ae0 con 0x7fe7940b52f0 2023-06-22T17:03:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe798c65700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7940b5dc0 con 0x7fe7940b24e0 2023-06-22T17:03:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe793fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7940b52f0 0x7fe7940cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe793fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7940b52f0 0x7fe7940cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56794/0 (socket says 172.21.15.47:56794) 2023-06-22T17:03:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe793fff700 1 -- 172.21.15.47:0/2149503495 learned_addr learned my addr 172.21.15.47:0/2149503495 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:03:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe792ffd700 1 --2- 172.21.15.47:0/2149503495 >> v2:172.21.15.202:3300/0 conn(0x7fe7940b24e0 0x7fe7940b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe793fff700 1 -- 172.21.15.47:0/2149503495 >> v2:172.21.15.202:3300/0 conn(0x7fe7940b24e0 msgr2=0x7fe7940b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe793fff700 1 --2- 172.21.15.47:0/2149503495 >> v2:172.21.15.202:3300/0 conn(0x7fe7940b24e0 0x7fe7940b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe793fff700 1 -- 172.21.15.47:0/2149503495 >> v1:172.21.15.202:6789/0 conn(0x7fe794142de0 legacy=0x7fe7940b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.948+0000 7fe793fff700 1 -- 172.21.15.47:0/2149503495 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940b60a0 con 0x7fe7940b52f0 2023-06-22T17:03:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe793fff700 1 --2- 172.21.15.47:0/2149503495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7940b52f0 0x7fe7940cf0b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7940af100 tx=0x7fe78c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cd5e4dc29f87a4 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe7927fc700 1 -- 172.21.15.47:0/2149503495 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00e070 con 0x7fe7940b52f0 2023-06-22T17:03:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe7927fc700 1 -- 172.21.15.47:0/2149503495 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c00ba10 con 0x7fe7940b52f0 2023-06-22T17:03:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe7927fc700 1 -- 172.21.15.47:0/2149503495 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0092e0 con 0x7fe7940b52f0 2023-06-22T17:03:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe798c65700 1 -- 172.21.15.47:0/2149503495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7940b52f0 msgr2=0x7fe7940cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe798c65700 1 --2- 172.21.15.47:0/2149503495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7940b52f0 0x7fe7940cf0b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7940af100 tx=0x7fe78c009840 comp rx=0 tx=0).stop 2023-06-22T17:03:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe798c65700 1 -- 172.21.15.47:0/2149503495 shutdown_connections 2023-06-22T17:03:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe798c65700 1 --2- 172.21.15.47:0/2149503495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7940b52f0 0x7fe7940cf0b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe798c65700 1 --2- 172.21.15.47:0/2149503495 >> v2:172.21.15.202:3300/0 conn(0x7fe7940b24e0 0x7fe7940b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe798c65700 1 -- 172.21.15.47:0/2149503495 >> 172.21.15.47:0/2149503495 conn(0x7fe794070290 msgr2=0x7fe794070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe798c65700 1 -- 172.21.15.47:0/2149503495 shutdown_connections 2023-06-22T17:03:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.949+0000 7fe798c65700 1 -- 172.21.15.47:0/2149503495 wait complete. 2023-06-22T17:03:50.955 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.951+0000 7fe798c65700 1 Processor -- start 2023-06-22T17:03:50.955 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.951+0000 7fe798c65700 1 -- start start 2023-06-22T17:03:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.951+0000 7fe798c65700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fe7940b24e0 0x7fe79416c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.951+0000 7fe798c65700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fe7940b52f0 0x7fe79415df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.951+0000 7fe798c65700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe794142de0 0x7fe79415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.951+0000 7fe798c65700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79415cad0 con 0x7fe794142de0 2023-06-22T17:03:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.951+0000 7fe798c65700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79415cdb0 con 0x7fe7940b24e0 2023-06-22T17:03:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.951+0000 7fe798c65700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79415d090 con 0x7fe7940b52f0 2023-06-22T17:03:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.951+0000 7fe793fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe794142de0 0x7fe79415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe793fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe794142de0 0x7fe79415e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56804/0 (socket says 172.21.15.47:56804) 2023-06-22T17:03:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe793fff700 1 -- 172.21.15.47:0/2835431081 learned_addr learned my addr 172.21.15.47:0/2835431081 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:03:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe7937fe700 1 --2- 172.21.15.47:0/2835431081 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fe7940b24e0 0x7fe79416c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe792ffd700 1 --2- 172.21.15.47:0/2835431081 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fe7940b52f0 0x7fe79415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe793fff700 1 -- 172.21.15.47:0/2835431081 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fe7940b52f0 msgr2=0x7fe79415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe793fff700 1 --2- 172.21.15.47:0/2835431081 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fe7940b52f0 0x7fe79415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe793fff700 1 -- 172.21.15.47:0/2835431081 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fe7940b24e0 msgr2=0x7fe79416c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe793fff700 1 --2- 172.21.15.47:0/2835431081 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fe7940b24e0 0x7fe79416c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe793fff700 1 -- 172.21.15.47:0/2835431081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c009ae0 con 0x7fe794142de0 2023-06-22T17:03:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe793fff700 1 --2- 172.21.15.47:0/2835431081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe794142de0 0x7fe79415e4d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe794067e10 tx=0x7fe78c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe78b7fe700 1 -- 172.21.15.47:0/2835431081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00bee0 con 0x7fe794142de0 2023-06-22T17:03:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe78b7fe700 1 -- 172.21.15.47:0/2835431081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c0032f0 con 0x7fe794142de0 2023-06-22T17:03:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe798c65700 1 -- 172.21.15.47:0/2835431081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79415d370 con 0x7fe794142de0 2023-06-22T17:03:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe78b7fe700 1 -- 172.21.15.47:0/2835431081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c01d420 con 0x7fe794142de0 2023-06-22T17:03:50.958 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe798c65700 1 -- 172.21.15.47:0/2835431081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79415bda0 con 0x7fe794142de0 2023-06-22T17:03:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe78b7fe700 1 -- 172.21.15.47:0/2835431081 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe78c01d850 con 0x7fe794142de0 2023-06-22T17:03:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.952+0000 7fe78b7fe700 1 -- 172.21.15.47:0/2835431081 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fe78c025b70 con 0x7fe794142de0 2023-06-22T17:03:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.953+0000 7fe798c65700 1 -- 172.21.15.47:0/2835431081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79416acd0 con 0x7fe794142de0 2023-06-22T17:03:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:50.955+0000 7fe78b7fe700 1 -- 172.21.15.47:0/2835431081 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fe79416acd0 con 0x7fe794142de0 2023-06-22T17:03:51.027 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.026+0000 7fe798c65700 1 -- 172.21.15.47:0/2835431081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe794088a10 con 0x7fe794142de0 2023-06-22T17:03:51.027 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.027+0000 7fe78b7fe700 1 -- 172.21.15.47:0/2835431081 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fe78c02a060 con 0x7fe794142de0 2023-06-22T17:03:51.028 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:03:51.028 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"fsid":"61e931aa-8dfe-4361-8a29-1d30432b403b","created":"2023-06-22T17:03:48.391484+0000","modified":"2023-06-22T17:03:50.761615+0000","last_up_change":"0.000000","last_in_change":"2023-06-22T17:03:50.761615+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"191ca9cf-dc97-4325-b63b-a3e6a66b71b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d426e8cd-b1e6-40a3-9e65-cf52fac95a21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"487a8fca-1da6-4cb3-a73c-abc6e9e2a29f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7089499f-dc69-4d6b-a6c3-5b30404ab1bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f8b34dee-7945-42fd-9e58-98e5fe196b36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9583b379-49e1-4b6c-adf5-6f19322cc76c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d11633b5-5a36-441b-93c7-7c0b434f0dbc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"edd9bf42-9566-4d07-902b-08cf16cb42a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T17:03:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.028+0000 7fe798c65700 1 -- 172.21.15.47:0/2835431081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe794142de0 msgr2=0x7fe79415e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.028+0000 7fe798c65700 1 --2- 172.21.15.47:0/2835431081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe794142de0 0x7fe79415e4d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe794067e10 tx=0x7fe78c017000 comp rx=0 tx=0).stop 2023-06-22T17:03:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.028+0000 7fe798c65700 1 -- 172.21.15.47:0/2835431081 shutdown_connections 2023-06-22T17:03:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.028+0000 7fe798c65700 1 --2- 172.21.15.47:0/2835431081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe794142de0 0x7fe79415e4d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.028+0000 7fe798c65700 1 --2- 172.21.15.47:0/2835431081 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fe7940b24e0 0x7fe79416c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.028+0000 7fe798c65700 1 --2- 172.21.15.47:0/2835431081 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fe7940b52f0 0x7fe79415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:51.030 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.028+0000 7fe798c65700 1 -- 172.21.15.47:0/2835431081 >> 172.21.15.47:0/2835431081 conn(0x7fe794070290 msgr2=0x7fe7940b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:51.030 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.028+0000 7fe798c65700 1 -- 172.21.15.47:0/2835431081 shutdown_connections 2023-06-22T17:03:51.030 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:03:51.028+0000 7fe798c65700 1 -- 172.21.15.47:0/2835431081 wait complete. 2023-06-22T17:03:51.040 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-06-22T17:03:51.041 INFO:tasks.ceph:Waiting for OSDs to come up 2023-06-22T17:03:51.131 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:51.130+0000 7f0769aba180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-22T17:03:51.213 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:51.213+0000 7f0769aba180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-22T17:03:51.249 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:51.248+0000 7fbc2e468180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-22T17:03:51.296 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:51.295+0000 7f0769aba180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-22T17:03:51.326 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:51.325+0000 7fbc2e468180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-22T17:03:51.394 INFO:tasks.ceph.osd.2.smithi047.stderr:2023-06-22T17:03:51.393+0000 7fe261dde540 -1 Falling back to public interface 2023-06-22T17:03:51.395 INFO:tasks.ceph.osd.0.smithi047.stderr:2023-06-22T17:03:51.394+0000 7ff87723b540 -1 Falling back to public interface 2023-06-22T17:03:51.397 INFO:tasks.ceph.osd.4.smithi202.stderr:2023-06-22T17:03:51.396+0000 7ff62dce4540 -1 Falling back to public interface 2023-06-22T17:03:51.398 INFO:tasks.ceph.osd.5.smithi202.stderr:2023-06-22T17:03:51.397+0000 7fad19897540 -1 Falling back to public interface 2023-06-22T17:03:51.399 INFO:tasks.ceph.osd.6.smithi202.stderr:2023-06-22T17:03:51.397+0000 7f0ae9dbc540 -1 Falling back to public interface 2023-06-22T17:03:51.402 INFO:tasks.ceph.osd.3.smithi047.stderr:2023-06-22T17:03:51.401+0000 7fe067d43540 -1 Falling back to public interface 2023-06-22T17:03:51.406 INFO:tasks.ceph.osd.7.smithi202.stderr:2023-06-22T17:03:51.405+0000 7fb1f5fcd540 -1 Falling back to public interface 2023-06-22T17:03:51.407 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:51.406+0000 7fbc2e468180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-22T17:03:51.412 INFO:tasks.ceph.osd.1.smithi047.stderr:2023-06-22T17:03:51.411+0000 7f5c1d249540 -1 Falling back to public interface 2023-06-22T17:03:51.847 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-22T17:03:51.913 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:51.912+0000 7f0769aba180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-22T17:03:51.958 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.957+0000 7f46e9433700 1 Processor -- start 2023-06-22T17:03:51.958 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.957+0000 7f46e9433700 1 -- start start 2023-06-22T17:03:51.964 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e9433700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f46e415cb30 0x7f46e4068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:51.964 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e9433700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f46e4063120 0x7f46e40683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:51.964 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e9433700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e40aec40 con 0x7f46e415cf00 2023-06-22T17:03:51.965 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e9433700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e4069520 con 0x7f46e4063120 2023-06-22T17:03:51.965 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e9433700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e4067f60 con 0x7f46e415cb30 2023-06-22T17:03:51.965 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e27fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f46e4063120 0x7f46e40683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:51.966 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e27fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f46e4063120 0x7f46e40683f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56810/0 (socket says 172.21.15.47:56810) 2023-06-22T17:03:51.966 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e27fc700 1 -- 172.21.15.47:0/1758738665 learned_addr learned my addr 172.21.15.47:0/1758738665 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:03:51.966 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e2ffd700 1 --2- 172.21.15.47:0/1758738665 >> v2:172.21.15.202:3300/0 conn(0x7f46e415cb30 0x7f46e4068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:51.966 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e27fc700 1 -- 172.21.15.47:0/1758738665 >> v2:172.21.15.202:3300/0 conn(0x7f46e415cb30 msgr2=0x7f46e4068fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:51.967 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e27fc700 1 --2- 172.21.15.47:0/1758738665 >> v2:172.21.15.202:3300/0 conn(0x7f46e415cb30 0x7f46e4068fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:51.967 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e27fc700 1 -- 172.21.15.47:0/1758738665 >> v1:172.21.15.202:6789/0 conn(0x7f46e415cf00 legacy=0x7f46e406dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:51.967 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e27fc700 1 -- 172.21.15.47:0/1758738665 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4067c30 con 0x7f46e4063120 2023-06-22T17:03:51.968 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e2ffd700 1 --2- 172.21.15.47:0/1758738665 >> v2:172.21.15.202:3300/0 conn(0x7f46e415cb30 0x7f46e4068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T17:03:51.968 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e27fc700 1 --2- 172.21.15.47:0/1758738665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e4063120 0x7f46e40683f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f46e4064cd0 tx=0x7f46d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b347d98a9719869f server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:51.968 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e1ffb700 1 -- 172.21.15.47:0/1758738665 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d800b860 con 0x7f46e4063120 2023-06-22T17:03:51.968 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e1ffb700 1 -- 172.21.15.47:0/1758738665 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d800b9c0 con 0x7f46e4063120 2023-06-22T17:03:51.969 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.958+0000 7f46e1ffb700 1 -- 172.21.15.47:0/1758738665 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d800bd60 con 0x7f46e4063120 2023-06-22T17:03:51.969 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.959+0000 7f46e9433700 1 -- 172.21.15.47:0/1758738665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e4063120 msgr2=0x7f46e40683f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:51.969 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.959+0000 7f46e9433700 1 --2- 172.21.15.47:0/1758738665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e4063120 0x7f46e40683f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f46e4064cd0 tx=0x7f46d8009250 comp rx=0 tx=0).stop 2023-06-22T17:03:51.970 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.959+0000 7f46e9433700 1 -- 172.21.15.47:0/1758738665 shutdown_connections 2023-06-22T17:03:51.970 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.959+0000 7f46e9433700 1 --2- 172.21.15.47:0/1758738665 >> v2:172.21.15.202:3300/0 conn(0x7f46e415cb30 0x7f46e4068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:51.971 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.959+0000 7f46e9433700 1 --2- 172.21.15.47:0/1758738665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e4063120 0x7f46e40683f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:51.971 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.959+0000 7f46e9433700 1 -- 172.21.15.47:0/1758738665 >> 172.21.15.47:0/1758738665 conn(0x7f46e406c150 msgr2=0x7f46e40ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:51.971 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.959+0000 7f46e9433700 1 -- 172.21.15.47:0/1758738665 shutdown_connections 2023-06-22T17:03:51.971 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.959+0000 7f46e9433700 1 -- 172.21.15.47:0/1758738665 wait complete. 2023-06-22T17:03:51.972 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.960+0000 7f46e9433700 1 Processor -- start 2023-06-22T17:03:51.972 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.960+0000 7f46e9433700 1 -- start start 2023-06-22T17:03:51.972 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e9433700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f46e4063120 0x7f46e41560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:51.973 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e9433700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e415cb30 0x7f46e413d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:51.973 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e9433700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f46e415cf00 0x7f46e4169db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:51.973 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e9433700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e416adc0 con 0x7f46e415cb30 2023-06-22T17:03:51.973 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e9433700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e41565e0 con 0x7f46e4063120 2023-06-22T17:03:51.974 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e9433700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e41567f0 con 0x7f46e415cf00 2023-06-22T17:03:51.974 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e27fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e415cb30 0x7f46e413d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:51.974 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e27fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e415cb30 0x7f46e413d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56816/0 (socket says 172.21.15.47:56816) 2023-06-22T17:03:51.974 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e27fc700 1 -- 172.21.15.47:0/2770485351 learned_addr learned my addr 172.21.15.47:0/2770485351 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:03:51.975 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e2ffd700 1 --2- 172.21.15.47:0/2770485351 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f46e4063120 0x7f46e41560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:51.975 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.961+0000 7f46e37fe700 1 --2- 172.21.15.47:0/2770485351 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f46e415cf00 0x7f46e4169db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:51.975 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46e27fc700 1 -- 172.21.15.47:0/2770485351 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f46e415cf00 msgr2=0x7f46e4169db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:51.976 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46e27fc700 1 --2- 172.21.15.47:0/2770485351 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f46e415cf00 0x7f46e4169db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:51.976 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46e27fc700 1 -- 172.21.15.47:0/2770485351 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f46e4063120 msgr2=0x7f46e41560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:51.976 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46e27fc700 1 --2- 172.21.15.47:0/2770485351 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f46e4063120 0x7f46e41560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:51.976 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46e27fc700 1 -- 172.21.15.47:0/2770485351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d80094f0 con 0x7f46e415cb30 2023-06-22T17:03:51.977 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46e27fc700 1 --2- 172.21.15.47:0/2770485351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e415cb30 0x7f46e413d3d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009290 tx=0x7f46d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:51.977 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46caffd700 1 -- 172.21.15.47:0/2770485351 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d801a450 con 0x7f46e415cb30 2023-06-22T17:03:51.977 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46caffd700 1 -- 172.21.15.47:0/2770485351 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d801a5b0 con 0x7f46e415cb30 2023-06-22T17:03:51.977 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46caffd700 1 -- 172.21.15.47:0/2770485351 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d801a970 con 0x7f46e415cb30 2023-06-22T17:03:51.977 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46e9433700 1 -- 172.21.15.47:0/2770485351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e4156ad0 con 0x7f46e415cb30 2023-06-22T17:03:51.978 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.962+0000 7f46e9433700 1 -- 172.21.15.47:0/2770485351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e4157090 con 0x7f46e415cb30 2023-06-22T17:03:51.978 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.963+0000 7f46caffd700 1 -- 172.21.15.47:0/2770485351 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f46d801b430 con 0x7f46e415cb30 2023-06-22T17:03:51.978 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.963+0000 7f46caffd700 1 -- 172.21.15.47:0/2770485351 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f46d8025af0 con 0x7f46e415cb30 2023-06-22T17:03:51.978 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.963+0000 7f46e9433700 1 -- 172.21.15.47:0/2770485351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e40ad830 con 0x7f46e415cb30 2023-06-22T17:03:51.978 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:51.965+0000 7f46caffd700 1 -- 172.21.15.47:0/2770485351 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f46d80446e0 con 0x7f46e415cb30 2023-06-22T17:03:52.006 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:52.004+0000 7fbc2e468180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-22T17:03:52.069 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.069+0000 7f46e9433700 1 -- 172.21.15.47:0/2770485351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46e404c1b0 con 0x7f46e415cb30 2023-06-22T17:03:52.070 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.069+0000 7f46caffd700 1 -- 172.21.15.47:0/2770485351 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f46d8033030 con 0x7f46e415cb30 2023-06-22T17:03:52.071 INFO:teuthology.misc.health.smithi047.stdout: 2023-06-22T17:03:52.071 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":10,"fsid":"61e931aa-8dfe-4361-8a29-1d30432b403b","created":"2023-06-22T17:03:48.391484+0000","modified":"2023-06-22T17:03:50.761615+0000","last_up_change":"0.000000","last_in_change":"2023-06-22T17:03:50.761615+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"191ca9cf-dc97-4325-b63b-a3e6a66b71b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d426e8cd-b1e6-40a3-9e65-cf52fac95a21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"487a8fca-1da6-4cb3-a73c-abc6e9e2a29f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7089499f-dc69-4d6b-a6c3-5b30404ab1bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f8b34dee-7945-42fd-9e58-98e5fe196b36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9583b379-49e1-4b6c-adf5-6f19322cc76c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d11633b5-5a36-441b-93c7-7c0b434f0dbc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"edd9bf42-9566-4d07-902b-08cf16cb42a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T17:03:52.072 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.070+0000 7f46e9433700 1 -- 172.21.15.47:0/2770485351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e415cb30 msgr2=0x7f46e413d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:52.072 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.070+0000 7f46e9433700 1 --2- 172.21.15.47:0/2770485351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e415cb30 0x7f46e413d3d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009290 tx=0x7f46d8017000 comp rx=0 tx=0).stop 2023-06-22T17:03:52.073 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.070+0000 7f46e9433700 1 -- 172.21.15.47:0/2770485351 shutdown_connections 2023-06-22T17:03:52.073 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.070+0000 7f46e9433700 1 --2- 172.21.15.47:0/2770485351 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f46e415cf00 0x7f46e4169db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:52.073 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.070+0000 7f46e9433700 1 --2- 172.21.15.47:0/2770485351 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f46e4063120 0x7f46e41560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:52.073 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.070+0000 7f46e9433700 1 --2- 172.21.15.47:0/2770485351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e415cb30 0x7f46e413d3d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:52.073 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.070+0000 7f46e9433700 1 -- 172.21.15.47:0/2770485351 >> 172.21.15.47:0/2770485351 conn(0x7f46e406c150 msgr2=0x7f46e415fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:52.074 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.070+0000 7f46e9433700 1 -- 172.21.15.47:0/2770485351 shutdown_connections 2023-06-22T17:03:52.074 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:52.070+0000 7f46e9433700 1 -- 172.21.15.47:0/2770485351 wait complete. 2023-06-22T17:03:52.082 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-06-22T17:03:52.089 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:52.088+0000 7f0769aba180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-22T17:03:52.142 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:52.141+0000 7fbc2e468180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-22T17:03:52.187 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:52.186+0000 7f0769aba180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-22T17:03:52.246 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:52.245+0000 7fbc2e468180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-22T17:03:52.446 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:52.445+0000 7f0769aba180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-22T17:03:52.497 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:52.496+0000 7fbc2e468180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-22T17:03:52.528 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:52.527+0000 7f0769aba180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-22T17:03:52.575 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:52.574+0000 7fbc2e468180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-22T17:03:53.397 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:53.395+0000 7f0769aba180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-22T17:03:53.410 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:53.409+0000 7fbc2e468180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-22T17:03:53.474 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:53.472+0000 7f0769aba180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-22T17:03:53.483 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:53.482+0000 7fbc2e468180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-22T17:03:53.553 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:53.552+0000 7f0769aba180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-22T17:03:53.558 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:53.556+0000 7fbc2e468180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-22T17:03:53.690 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:53.689+0000 7fbc2e468180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-22T17:03:53.694 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:53.692+0000 7f0769aba180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-22T17:03:53.775 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:53.773+0000 7fbc2e468180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-22T17:03:53.782 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:53.781+0000 7f0769aba180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-22T17:03:53.933 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:53.932+0000 7fbc2e468180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-22T17:03:53.948 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:53.946+0000 7f0769aba180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-22T17:03:54.010 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:54.009+0000 7fbc2e468180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-22T17:03:54.028 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:54.026+0000 7f0769aba180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-22T17:03:54.086 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:54.084+0000 7fbc2e468180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-22T17:03:54.106 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:54.105+0000 7f0769aba180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-22T17:03:54.178 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:54.177+0000 7fbc2e468180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-22T17:03:54.201 INFO:tasks.ceph.mgr.y.smithi047.stderr:2023-06-22T17:03:54.199+0000 7f0769aba180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-22T17:03:54.917 INFO:tasks.ceph.osd.1.smithi047.stderr:2023-06-22T17:03:54.915+0000 7f5c1d249540 -1 osd.1 0 log_to_monitors true 2023-06-22T17:03:54.924 INFO:tasks.ceph.osd.3.smithi047.stderr:2023-06-22T17:03:54.922+0000 7fe067d43540 -1 osd.3 0 log_to_monitors true 2023-06-22T17:03:54.928 INFO:tasks.ceph.osd.5.smithi202.stderr:2023-06-22T17:03:54.927+0000 7fad19897540 -1 osd.5 0 log_to_monitors true 2023-06-22T17:03:54.929 INFO:tasks.ceph.osd.6.smithi202.stderr:2023-06-22T17:03:54.927+0000 7f0ae9dbc540 -1 osd.6 0 log_to_monitors true 2023-06-22T17:03:54.929 INFO:tasks.ceph.osd.2.smithi047.stderr:2023-06-22T17:03:54.927+0000 7fe261dde540 -1 osd.2 0 log_to_monitors true 2023-06-22T17:03:54.936 INFO:tasks.ceph.osd.0.smithi047.stderr:2023-06-22T17:03:54.935+0000 7ff87723b540 -1 osd.0 0 log_to_monitors true 2023-06-22T17:03:54.948 INFO:tasks.ceph.osd.4.smithi202.stderr:2023-06-22T17:03:54.947+0000 7ff62dce4540 -1 osd.4 0 log_to_monitors true 2023-06-22T17:03:54.948 INFO:tasks.ceph.osd.7.smithi202.stderr:2023-06-22T17:03:54.947+0000 7fb1f5fcd540 -1 osd.7 0 log_to_monitors true 2023-06-22T17:03:55.206 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:55.205+0000 7fbbe7869700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-06-22T17:03:55.207 INFO:tasks.ceph.mgr.x.smithi202.stderr:2023-06-22T17:03:55.205+0000 7fbbe7869700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-06-22T17:03:57.633 INFO:tasks.ceph.osd.2.smithi047.stderr:2023-06-22T17:03:57.631+0000 7fe258a00700 -1 osd.2 0 waiting for initial osdmap 2023-06-22T17:03:57.638 INFO:tasks.ceph.osd.2.smithi047.stderr:2023-06-22T17:03:57.636+0000 7fe2517f7700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T17:03:57.655 INFO:tasks.ceph.osd.1.smithi047.stderr:2023-06-22T17:03:57.653+0000 7f5c13e6b700 -1 osd.1 0 waiting for initial osdmap 2023-06-22T17:03:57.660 INFO:tasks.ceph.osd.1.smithi047.stderr:2023-06-22T17:03:57.658+0000 7f5c0cc62700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T17:03:57.669 INFO:tasks.ceph.osd.0.smithi047.stderr:2023-06-22T17:03:57.667+0000 7ff86de5d700 -1 osd.0 0 waiting for initial osdmap 2023-06-22T17:03:57.672 INFO:tasks.ceph.osd.0.smithi047.stderr:2023-06-22T17:03:57.670+0000 7ff866c54700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T17:03:57.712 INFO:tasks.ceph.osd.3.smithi047.stderr:2023-06-22T17:03:57.710+0000 7fe05e965700 -1 osd.3 0 waiting for initial osdmap 2023-06-22T17:03:57.716 INFO:tasks.ceph.osd.3.smithi047.stderr:2023-06-22T17:03:57.714+0000 7fe05775c700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T17:03:58.341 INFO:tasks.ceph.osd.4.smithi202.stderr:2023-06-22T17:03:58.339+0000 7ff624906700 -1 osd.4 0 waiting for initial osdmap 2023-06-22T17:03:58.349 INFO:tasks.ceph.osd.4.smithi202.stderr:2023-06-22T17:03:58.347+0000 7ff61d6fd700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T17:03:58.356 INFO:tasks.ceph.osd.5.smithi202.stderr:2023-06-22T17:03:58.355+0000 7fad104b9700 -1 osd.5 0 waiting for initial osdmap 2023-06-22T17:03:58.357 INFO:tasks.ceph.osd.7.smithi202.stderr:2023-06-22T17:03:58.355+0000 7fb1ecbef700 -1 osd.7 0 waiting for initial osdmap 2023-06-22T17:03:58.362 INFO:tasks.ceph.osd.5.smithi202.stderr:2023-06-22T17:03:58.361+0000 7fad092b0700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T17:03:58.362 INFO:tasks.ceph.osd.7.smithi202.stderr:2023-06-22T17:03:58.361+0000 7fb1e59e6700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T17:03:58.383 INFO:tasks.ceph.osd.6.smithi202.stderr:2023-06-22T17:03:58.378+0000 7f0ae09de700 -1 osd.6 0 waiting for initial osdmap 2023-06-22T17:03:58.384 INFO:tasks.ceph.osd.6.smithi202.stderr:2023-06-22T17:03:58.382+0000 7f0ad97d5700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T17:03:58.887 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-22T17:03:59.008 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.006+0000 7fc43d071700 1 Processor -- start 2023-06-22T17:03:59.008 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.006+0000 7fc43d071700 1 -- start start 2023-06-22T17:03:59.011 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43d071700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4380ceb50 0x7fc4380b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:59.012 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43d071700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fc4380cbc80 0x7fc4380cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:59.012 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43d071700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4380cb250 con 0x7fc4380ccea0 2023-06-22T17:03:59.012 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43d071700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc438063bc0 con 0x7fc4380ceb50 2023-06-22T17:03:59.012 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43d071700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4380b42c0 con 0x7fc4380cbc80 2023-06-22T17:03:59.013 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4380ceb50 0x7fc4380b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:59.013 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43659c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4380ceb50 0x7fc4380b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33830/0 (socket says 172.21.15.47:33830) 2023-06-22T17:03:59.013 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43659c700 1 -- 172.21.15.47:0/3358662837 learned_addr learned my addr 172.21.15.47:0/3358662837 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:03:59.013 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43759e700 1 --2- 172.21.15.47:0/3358662837 >> v2:172.21.15.202:3300/0 conn(0x7fc4380cbc80 0x7fc4380cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:59.014 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43659c700 1 -- 172.21.15.47:0/3358662837 >> v2:172.21.15.202:3300/0 conn(0x7fc4380cbc80 msgr2=0x7fc4380cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:03:59.014 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43659c700 1 --2- 172.21.15.47:0/3358662837 >> v2:172.21.15.202:3300/0 conn(0x7fc4380cbc80 0x7fc4380cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:59.014 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43659c700 1 -- 172.21.15.47:0/3358662837 >> v1:172.21.15.202:6789/0 conn(0x7fc4380ccea0 legacy=0x7fc4380cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:59.015 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43659c700 1 -- 172.21.15.47:0/3358662837 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380cb7f0 con 0x7fc4380ceb50 2023-06-22T17:03:59.015 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc43659c700 1 --2- 172.21.15.47:0/3358662837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 0x7fc4380b3d80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4380899a0 tx=0x7fc420009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2058e947d913a9a5 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:59.015 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc435d9b700 1 -- 172.21.15.47:0/3358662837 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42000b860 con 0x7fc4380ceb50 2023-06-22T17:03:59.015 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc435d9b700 1 -- 172.21.15.47:0/3358662837 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42000b9c0 con 0x7fc4380ceb50 2023-06-22T17:03:59.016 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.007+0000 7fc435d9b700 1 -- 172.21.15.47:0/3358662837 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42000bd60 con 0x7fc4380ceb50 2023-06-22T17:03:59.016 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.008+0000 7fc43d071700 1 -- 172.21.15.47:0/3358662837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 msgr2=0x7fc4380b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:59.016 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.008+0000 7fc43d071700 1 --2- 172.21.15.47:0/3358662837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 0x7fc4380b3d80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4380899a0 tx=0x7fc420009250 comp rx=0 tx=0).stop 2023-06-22T17:03:59.017 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.008+0000 7fc43d071700 1 -- 172.21.15.47:0/3358662837 shutdown_connections 2023-06-22T17:03:59.017 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.008+0000 7fc43d071700 1 --2- 172.21.15.47:0/3358662837 >> v2:172.21.15.202:3300/0 conn(0x7fc4380cbc80 0x7fc4380cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:59.017 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.008+0000 7fc43d071700 1 --2- 172.21.15.47:0/3358662837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 0x7fc4380b3d80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:59.018 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.008+0000 7fc43d071700 1 -- 172.21.15.47:0/3358662837 >> 172.21.15.47:0/3358662837 conn(0x7fc438099870 msgr2=0x7fc4380b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:59.018 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.008+0000 7fc43d071700 1 -- 172.21.15.47:0/3358662837 shutdown_connections 2023-06-22T17:03:59.018 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.008+0000 7fc43d071700 1 -- 172.21.15.47:0/3358662837 wait complete. 2023-06-22T17:03:59.019 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.009+0000 7fc43d071700 1 Processor -- start 2023-06-22T17:03:59.019 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.009+0000 7fc43d071700 1 -- start start 2023-06-22T17:03:59.023 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.010+0000 7fc43d071700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fc4380cbc80 0x7fc438068fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:59.023 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.010+0000 7fc43d071700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fc4380ccea0 0x7fc43815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:59.023 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.010+0000 7fc43d071700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 0x7fc438169f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:59.024 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.010+0000 7fc43d071700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43815eb60 con 0x7fc4380ceb50 2023-06-22T17:03:59.024 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.010+0000 7fc43d071700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43815e020 con 0x7fc4380cbc80 2023-06-22T17:03:59.024 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.010+0000 7fc43d071700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43815e2c0 con 0x7fc4380ccea0 2023-06-22T17:03:59.024 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.010+0000 7fc43759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 0x7fc438169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:59.025 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.010+0000 7fc43759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 0x7fc438169f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33834/0 (socket says 172.21.15.47:33834) 2023-06-22T17:03:59.025 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.010+0000 7fc43759e700 1 -- 172.21.15.47:0/2208913492 learned_addr learned my addr 172.21.15.47:0/2208913492 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:03:59.025 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.010+0000 7fc436d9d700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fc4380cbc80 0x7fc438068fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:59.025 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc43659c700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fc4380ccea0 0x7fc43815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:59.026 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc43759e700 1 -- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fc4380ccea0 msgr2=0x7fc43815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:59.026 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc43759e700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fc4380ccea0 0x7fc43815dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:59.026 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc43759e700 1 -- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fc4380cbc80 msgr2=0x7fc438068fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:59.027 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc43759e700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fc4380cbc80 0x7fc438068fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:59.027 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc43759e700 1 -- 172.21.15.47:0/2208913492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4200094f0 con 0x7fc4380ceb50 2023-06-22T17:03:59.027 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc436d9d700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fc4380cbc80 0x7fc438068fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-22T17:03:59.027 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc43759e700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 0x7fc438169f00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc430009fa0 tx=0x7fc430009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:59.028 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc42effd700 1 -- 172.21.15.47:0/2208913492 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000ed20 con 0x7fc4380ceb50 2023-06-22T17:03:59.028 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc42effd700 1 -- 172.21.15.47:0/2208913492 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43000ee80 con 0x7fc4380ceb50 2023-06-22T17:03:59.028 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc43d071700 1 -- 172.21.15.47:0/2208913492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43815e5a0 con 0x7fc4380ceb50 2023-06-22T17:03:59.029 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc42effd700 1 -- 172.21.15.47:0/2208913492 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc430014670 con 0x7fc4380ceb50 2023-06-22T17:03:59.029 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.011+0000 7fc43d071700 1 -- 172.21.15.47:0/2208913492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4380c7780 con 0x7fc4380ceb50 2023-06-22T17:03:59.029 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.012+0000 7fc43d071700 1 -- 172.21.15.47:0/2208913492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4380c7970 con 0x7fc4380ceb50 2023-06-22T17:03:59.029 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.012+0000 7fc42effd700 1 -- 172.21.15.47:0/2208913492 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7fc430014a20 con 0x7fc4380ceb50 2023-06-22T17:03:59.030 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.013+0000 7fc42effd700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fc418066e00 0x7fc4180692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:03:59.030 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.013+0000 7fc42effd700 1 -- 172.21.15.47:0/2208913492 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3347+0+0 (secure 0 0 0) 0x7fc430084130 con 0x7fc4380ceb50 2023-06-22T17:03:59.030 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.013+0000 7fc436d9d700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fc418066e00 0x7fc4180692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:03:59.030 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.013+0000 7fc436d9d700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fc418066e00 0x7fc4180692b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc428001840 tx=0x7fc428009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:03:59.030 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.017+0000 7fc42effd700 1 -- 172.21.15.47:0/2208913492 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fc430051e70 con 0x7fc4380ceb50 2023-06-22T17:03:59.214 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.213+0000 7fc43d071700 1 -- 172.21.15.47:0/2208913492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc43804c1b0 con 0x7fc4380ceb50 2023-06-22T17:03:59.215 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.214+0000 7fc42effd700 1 -- 172.21.15.47:0/2208913492 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8692 (secure 0 0 0) 0x7fc430009990 con 0x7fc4380ceb50 2023-06-22T17:03:59.216 INFO:teuthology.misc.health.smithi047.stdout: 2023-06-22T17:03:59.216 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":13,"fsid":"61e931aa-8dfe-4361-8a29-1d30432b403b","created":"2023-06-22T17:03:48.391484+0000","modified":"2023-06-22T17:03:58.361240+0000","last_up_change":"2023-06-22T17:03:58.361240+0000","last_in_change":"2023-06-22T17:03:50.761615+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"191ca9cf-dc97-4325-b63b-a3e6a66b71b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":87966},{"type":"v1","addr":"172.21.15.47:6809","nonce":87966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":87966},{"type":"v1","addr":"172.21.15.47:6811","nonce":87966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":87966},{"type":"v1","addr":"172.21.15.47:6815","nonce":87966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":87966},{"type":"v1","addr":"172.21.15.47:6813","nonce":87966}]},"public_addr":"172.21.15.47:6809/87966","cluster_addr":"172.21.15.47:6811/87966","heartbeat_back_addr":"172.21.15.47:6815/87966","heartbeat_front_addr":"172.21.15.47:6813/87966","state":["exists","up"]},{"osd":1,"uuid":"d426e8cd-b1e6-40a3-9e65-cf52fac95a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":87978},{"type":"v1","addr":"172.21.15.47:6825","nonce":87978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":87978},{"type":"v1","addr":"172.21.15.47:6827","nonce":87978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":87978},{"type":"v1","addr":"172.21.15.47:6831","nonce":87978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":87978},{"type":"v1","addr":"172.21.15.47:6829","nonce":87978}]},"public_addr":"172.21.15.47:6825/87978","cluster_addr":"172.21.15.47:6827/87978","heartbeat_back_addr":"172.21.15.47:6831/87978","heartbeat_front_addr":"172.21.15.47:6829/87978","state":["exists","up"]},{"osd":2,"uuid":"487a8fca-1da6-4cb3-a73c-abc6e9e2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":87967},{"type":"v1","addr":"172.21.15.47:6801","nonce":87967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":87967},{"type":"v1","addr":"172.21.15.47:6803","nonce":87967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":87967},{"type":"v1","addr":"172.21.15.47:6807","nonce":87967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":87967},{"type":"v1","addr":"172.21.15.47:6805","nonce":87967}]},"public_addr":"172.21.15.47:6801/87967","cluster_addr":"172.21.15.47:6803/87967","heartbeat_back_addr":"172.21.15.47:6807/87967","heartbeat_front_addr":"172.21.15.47:6805/87967","state":["exists","up"]},{"osd":3,"uuid":"7089499f-dc69-4d6b-a6c3-5b30404ab1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":87968},{"type":"v1","addr":"172.21.15.47:6817","nonce":87968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":87968},{"type":"v1","addr":"172.21.15.47:6819","nonce":87968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":87968},{"type":"v1","addr":"172.21.15.47:6823","nonce":87968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":87968},{"type":"v1","addr":"172.21.15.47:6821","nonce":87968}]},"public_addr":"172.21.15.47:6817/87968","cluster_addr":"172.21.15.47:6819/87968","heartbeat_back_addr":"172.21.15.47:6823/87968","heartbeat_front_addr":"172.21.15.47:6821/87968","state":["exists","up"]},{"osd":4,"uuid":"f8b34dee-7945-42fd-9e58-98e5fe196b36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9583b379-49e1-4b6c-adf5-6f19322cc76c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d11633b5-5a36-441b-93c7-7c0b434f0dbc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"edd9bf42-9566-4d07-902b-08cf16cb42a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T17:03:59.218 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 -- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fc418066e00 msgr2=0x7fc4180692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:59.218 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fc418066e00 0x7fc4180692b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc428001840 tx=0x7fc428009040 comp rx=0 tx=0).stop 2023-06-22T17:03:59.218 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 -- 172.21.15.47:0/2208913492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 msgr2=0x7fc438169f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:03:59.218 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 0x7fc438169f00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc430009fa0 tx=0x7fc430009900 comp rx=0 tx=0).stop 2023-06-22T17:03:59.218 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 -- 172.21.15.47:0/2208913492 shutdown_connections 2023-06-22T17:03:59.219 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fc418066e00 0x7fc4180692b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:59.219 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4380ceb50 0x7fc438169f00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:59.219 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fc4380cbc80 0x7fc438068fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:59.219 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 --2- 172.21.15.47:0/2208913492 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fc4380ccea0 0x7fc43815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:03:59.219 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 -- 172.21.15.47:0/2208913492 >> 172.21.15.47:0/2208913492 conn(0x7fc438099870 msgr2=0x7fc43815c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:03:59.219 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 -- 172.21.15.47:0/2208913492 shutdown_connections 2023-06-22T17:03:59.220 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:03:59.216+0000 7fc43d071700 1 -- 172.21.15.47:0/2208913492 wait complete. 2023-06-22T17:03:59.228 DEBUG:teuthology.misc:4 of 8 OSDs are up 2023-06-22T17:04:06.035 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-22T17:04:06.158 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.156+0000 7f760b114700 1 Processor -- start 2023-06-22T17:04:06.159 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.156+0000 7f760b114700 1 -- start start 2023-06-22T17:04:06.160 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.157+0000 7f760b114700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f76040ceb50 0x7f76040b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.161 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.157+0000 7f760b114700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f76040cbc80 0x7f76040cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.161 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.157+0000 7f760b114700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76040cb250 con 0x7f76040ccea0 2023-06-22T17:04:06.161 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.157+0000 7f760b114700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7604063bc0 con 0x7f76040cbc80 2023-06-22T17:04:06.161 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.157+0000 7f760b114700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76040b42c0 con 0x7f76040ceb50 2023-06-22T17:04:06.161 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.157+0000 7f7608eb0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f76040ccea0 0x7f76040cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50790/0 (socket says 172.21.15.47:50790) 2023-06-22T17:04:06.161 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.157+0000 7f7608eb0700 1 -- 172.21.15.47:0/2881877663 learned_addr learned my addr 172.21.15.47:0/2881877663 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:06.162 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.157+0000 7f76096b1700 1 --2- 172.21.15.47:0/2881877663 >> v2:172.21.15.47:3300/0 conn(0x7f76040cbc80 0x7f76040cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.162 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f7603fff700 1 --2- 172.21.15.47:0/2881877663 >> v2:172.21.15.202:3300/0 conn(0x7f76040ceb50 0x7f76040b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.162 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f76037fe700 1 -- 172.21.15.47:0/2881877663 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271737220 0 0) 0x7f76040cb250 con 0x7f76040ccea0 2023-06-22T17:04:06.162 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f76037fe700 1 -- 172.21.15.47:0/2881877663 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75ec003570 con 0x7f76040ccea0 2023-06-22T17:04:06.162 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f76096b1700 1 -- 172.21.15.47:0/2881877663 >> v2:172.21.15.202:3300/0 conn(0x7f76040ceb50 msgr2=0x7f76040b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:06.162 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f76096b1700 1 --2- 172.21.15.47:0/2881877663 >> v2:172.21.15.202:3300/0 conn(0x7f76040ceb50 0x7f76040b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.162 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f76096b1700 1 -- 172.21.15.47:0/2881877663 >> v1:172.21.15.47:6789/0 conn(0x7f76040ccea0 legacy=0x7f76040cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:06.163 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f76096b1700 1 -- 172.21.15.47:0/2881877663 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76040cb7f0 con 0x7f76040cbc80 2023-06-22T17:04:06.163 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f76096b1700 1 --2- 172.21.15.47:0/2881877663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040cbc80 0x7f76040cc100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7604099fa0 tx=0x7f75fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606c8e2f6c24d1bc server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:06.163 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f76037fe700 1 -- 172.21.15.47:0/2881877663 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc00e070 con 0x7f76040cbc80 2023-06-22T17:04:06.163 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f76037fe700 1 -- 172.21.15.47:0/2881877663 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75fc00bab0 con 0x7f76040cbc80 2023-06-22T17:04:06.163 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.158+0000 7f76037fe700 1 -- 172.21.15.47:0/2881877663 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc00be70 con 0x7f76040cbc80 2023-06-22T17:04:06.163 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.159+0000 7f760b114700 1 -- 172.21.15.47:0/2881877663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040cbc80 msgr2=0x7f76040cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:06.163 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.159+0000 7f760b114700 1 --2- 172.21.15.47:0/2881877663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040cbc80 0x7f76040cc100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7604099fa0 tx=0x7f75fc009780 comp rx=0 tx=0).stop 2023-06-22T17:04:06.163 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.159+0000 7f760b114700 1 -- 172.21.15.47:0/2881877663 shutdown_connections 2023-06-22T17:04:06.164 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.159+0000 7f760b114700 1 --2- 172.21.15.47:0/2881877663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040cbc80 0x7f76040cc100 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.164 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.159+0000 7f760b114700 1 --2- 172.21.15.47:0/2881877663 >> v2:172.21.15.202:3300/0 conn(0x7f76040ceb50 0x7f76040b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.164 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.159+0000 7f760b114700 1 -- 172.21.15.47:0/2881877663 >> 172.21.15.47:0/2881877663 conn(0x7f7604099870 msgr2=0x7f76040b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:06.164 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.159+0000 7f760b114700 1 -- 172.21.15.47:0/2881877663 shutdown_connections 2023-06-22T17:04:06.164 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.159+0000 7f760b114700 1 -- 172.21.15.47:0/2881877663 wait complete. 2023-06-22T17:04:06.166 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.160+0000 7f760b114700 1 Processor -- start 2023-06-22T17:04:06.166 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.160+0000 7f760b114700 1 -- start start 2023-06-22T17:04:06.167 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f760b114700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f76040cbc80 0x7f7604150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.167 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f760b114700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f76040ccea0 0x7f760416a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.167 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f760b114700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040ceb50 0x7f76040c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.167 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f760b114700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760415bd20 con 0x7f76040ceb50 2023-06-22T17:04:06.167 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f760b114700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76040c7650 con 0x7f76040cbc80 2023-06-22T17:04:06.167 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f760b114700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76040c7c10 con 0x7f76040ccea0 2023-06-22T17:04:06.167 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f76096b1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040ceb50 0x7f76040c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.168 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f76096b1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040ceb50 0x7f76040c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33856/0 (socket says 172.21.15.47:33856) 2023-06-22T17:04:06.168 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f76096b1700 1 -- 172.21.15.47:0/4269410593 learned_addr learned my addr 172.21.15.47:0/4269410593 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:06.168 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f7608eb0700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f76040cbc80 0x7f7604150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.168 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f76096b1700 1 -- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f76040ccea0 msgr2=0x7f760416a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:06.168 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f76096b1700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f76040ccea0 0x7f760416a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.168 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f76096b1700 1 -- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f76040cbc80 msgr2=0x7f7604150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:06.169 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f76096b1700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f76040cbc80 0x7f7604150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.169 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f76096b1700 1 -- 172.21.15.47:0/4269410593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc009a20 con 0x7f76040ceb50 2023-06-22T17:04:06.169 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f7608eb0700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f76040cbc80 0x7f7604150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-22T17:04:06.169 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f76096b1700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040ceb50 0x7f76040c80a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0097c0 tx=0x7f75fc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:06.169 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.161+0000 7f7600ff9700 1 -- 172.21.15.47:0/4269410593 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc012e60 con 0x7f76040ceb50 2023-06-22T17:04:06.169 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.162+0000 7f7600ff9700 1 -- 172.21.15.47:0/4269410593 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75fc027470 con 0x7f76040ceb50 2023-06-22T17:04:06.169 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.162+0000 7f760b114700 1 -- 172.21.15.47:0/4269410593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760415a960 con 0x7f76040ceb50 2023-06-22T17:04:06.169 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.162+0000 7f7600ff9700 1 -- 172.21.15.47:0/4269410593 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc027830 con 0x7f76040ceb50 2023-06-22T17:04:06.170 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.162+0000 7f760b114700 1 -- 172.21.15.47:0/4269410593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76040c78a0 con 0x7f76040ceb50 2023-06-22T17:04:06.170 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.163+0000 7f760b114700 1 -- 172.21.15.47:0/4269410593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760404c1b0 con 0x7f76040ceb50 2023-06-22T17:04:06.170 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.163+0000 7f7600ff9700 1 -- 172.21.15.47:0/4269410593 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f75fc024070 con 0x7f76040ceb50 2023-06-22T17:04:06.170 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.164+0000 7f7600ff9700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f75f0066e90 0x7f75f0069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.170 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.164+0000 7f7600ff9700 1 -- 172.21.15.47:0/4269410593 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f75fc0867b0 con 0x7f76040ceb50 2023-06-22T17:04:06.170 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.164+0000 7f7608eb0700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f75f0066e90 0x7f75f0069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.170 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.164+0000 7f7608eb0700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f75f0066e90 0x7f75f0069340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f75f4007fd0 tx=0x7f75f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:06.171 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.168+0000 7f7600ff9700 1 -- 172.21.15.47:0/4269410593 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f75fc053d90 con 0x7f76040ceb50 2023-06-22T17:04:06.366 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.365+0000 7f760b114700 1 -- 172.21.15.47:0/4269410593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76040c7d50 con 0x7f76040ceb50 2023-06-22T17:04:06.367 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.365+0000 7f7600ff9700 1 -- 172.21.15.47:0/4269410593 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12127 (secure 0 0 0) 0x7f75fc0573b0 con 0x7f76040ceb50 2023-06-22T17:04:06.368 INFO:teuthology.misc.health.smithi047.stdout: 2023-06-22T17:04:06.368 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":17,"fsid":"61e931aa-8dfe-4361-8a29-1d30432b403b","created":"2023-06-22T17:03:48.391484+0000","modified":"2023-06-22T17:04:02.392828+0000","last_up_change":"2023-06-22T17:03:59.369744+0000","last_in_change":"2023-06-22T17:03:50.761615+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-22T17:04:00.277500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"191ca9cf-dc97-4325-b63b-a3e6a66b71b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":87966},{"type":"v1","addr":"172.21.15.47:6809","nonce":87966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":87966},{"type":"v1","addr":"172.21.15.47:6811","nonce":87966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":87966},{"type":"v1","addr":"172.21.15.47:6815","nonce":87966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":87966},{"type":"v1","addr":"172.21.15.47:6813","nonce":87966}]},"public_addr":"172.21.15.47:6809/87966","cluster_addr":"172.21.15.47:6811/87966","heartbeat_back_addr":"172.21.15.47:6815/87966","heartbeat_front_addr":"172.21.15.47:6813/87966","state":["exists","up"]},{"osd":1,"uuid":"d426e8cd-b1e6-40a3-9e65-cf52fac95a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":87978},{"type":"v1","addr":"172.21.15.47:6825","nonce":87978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":87978},{"type":"v1","addr":"172.21.15.47:6827","nonce":87978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":87978},{"type":"v1","addr":"172.21.15.47:6831","nonce":87978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":87978},{"type":"v1","addr":"172.21.15.47:6829","nonce":87978}]},"public_addr":"172.21.15.47:6825/87978","cluster_addr":"172.21.15.47:6827/87978","heartbeat_back_addr":"172.21.15.47:6831/87978","heartbeat_front_addr":"172.21.15.47:6829/87978","state":["exists","up"]},{"osd":2,"uuid":"487a8fca-1da6-4cb3-a73c-abc6e9e2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":87967},{"type":"v1","addr":"172.21.15.47:6801","nonce":87967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":87967},{"type":"v1","addr":"172.21.15.47:6803","nonce":87967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":87967},{"type":"v1","addr":"172.21.15.47:6807","nonce":87967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":87967},{"type":"v1","addr":"172.21.15.47:6805","nonce":87967}]},"public_addr":"172.21.15.47:6801/87967","cluster_addr":"172.21.15.47:6803/87967","heartbeat_back_addr":"172.21.15.47:6807/87967","heartbeat_front_addr":"172.21.15.47:6805/87967","state":["exists","up"]},{"osd":3,"uuid":"7089499f-dc69-4d6b-a6c3-5b30404ab1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":87968},{"type":"v1","addr":"172.21.15.47:6817","nonce":87968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":87968},{"type":"v1","addr":"172.21.15.47:6819","nonce":87968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":87968},{"type":"v1","addr":"172.21.15.47:6823","nonce":87968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":87968},{"type":"v1","addr":"172.21.15.47:6821","nonce":87968}]},"public_addr":"172.21.15.47:6817/87968","cluster_addr":"172.21.15.47:6819/87968","heartbeat_back_addr":"172.21.15.47:6823/87968","heartbeat_front_addr":"172.21.15.47:6821/87968","state":["exists","up"]},{"osd":4,"uuid":"f8b34dee-7945-42fd-9e58-98e5fe196b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6800","nonce":88201},{"type":"v1","addr":"172.21.15.202:6801","nonce":88201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6802","nonce":88201},{"type":"v1","addr":"172.21.15.202:6803","nonce":88201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6806","nonce":88201},{"type":"v1","addr":"172.21.15.202:6807","nonce":88201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6804","nonce":88201},{"type":"v1","addr":"172.21.15.202:6805","nonce":88201}]},"public_addr":"172.21.15.202:6801/88201","cluster_addr":"172.21.15.202:6803/88201","heartbeat_back_addr":"172.21.15.202:6807/88201","heartbeat_front_addr":"172.21.15.202:6805/88201","state":["exists","up"]},{"osd":5,"uuid":"9583b379-49e1-4b6c-adf5-6f19322cc76c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6808","nonce":88199},{"type":"v1","addr":"172.21.15.202:6809","nonce":88199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6810","nonce":88199},{"type":"v1","addr":"172.21.15.202:6812","nonce":88199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6818","nonce":88199},{"type":"v1","addr":"172.21.15.202:6820","nonce":88199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6814","nonce":88199},{"type":"v1","addr":"172.21.15.202:6816","nonce":88199}]},"public_addr":"172.21.15.202:6809/88199","cluster_addr":"172.21.15.202:6812/88199","heartbeat_back_addr":"172.21.15.202:6820/88199","heartbeat_front_addr":"172.21.15.202:6816/88199","state":["exists","up"]},{"osd":6,"uuid":"d11633b5-5a36-441b-93c7-7c0b434f0dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6811","nonce":88200},{"type":"v1","addr":"172.21.15.202:6813","nonce":88200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6815","nonce":88200},{"type":"v1","addr":"172.21.15.202:6817","nonce":88200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6822","nonce":88200},{"type":"v1","addr":"172.21.15.202:6823","nonce":88200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6819","nonce":88200},{"type":"v1","addr":"172.21.15.202:6821","nonce":88200}]},"public_addr":"172.21.15.202:6813/88200","cluster_addr":"172.21.15.202:6817/88200","heartbeat_back_addr":"172.21.15.202:6823/88200","heartbeat_front_addr":"172.21.15.202:6821/88200","state":["exists","up"]},{"osd":7,"uuid":"edd9bf42-9566-4d07-902b-08cf16cb42a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6824","nonce":88230},{"type":"v1","addr":"172.21.15.202:6825","nonce":88230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6826","nonce":88230},{"type":"v1","addr":"172.21.15.202:6827","nonce":88230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6830","nonce":88230},{"type":"v1","addr":"172.21.15.202:6831","nonce":88230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6828","nonce":88230},{"type":"v1","addr":"172.21.15.202:6829","nonce":88230}]},"public_addr":"172.21.15.202:6825/88230","cluster_addr":"172.21.15.202:6827/88230","heartbeat_back_addr":"172.21.15.202:6831/88230","heartbeat_front_addr":"172.21.15.202:6829/88230","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.975830+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.942848+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.958568+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.972986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.946508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.938793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.960236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.931356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T17:04:06.370 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 -- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f75f0066e90 msgr2=0x7f75f0069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:06.370 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f75f0066e90 0x7f75f0069340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f75f4007fd0 tx=0x7f75f4009040 comp rx=0 tx=0).stop 2023-06-22T17:04:06.370 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 -- 172.21.15.47:0/4269410593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040ceb50 msgr2=0x7f76040c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:06.370 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040ceb50 0x7f76040c80a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0097c0 tx=0x7f75fc019000 comp rx=0 tx=0).stop 2023-06-22T17:04:06.370 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 -- 172.21.15.47:0/4269410593 shutdown_connections 2023-06-22T17:04:06.370 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f75f0066e90 0x7f75f0069340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.371 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76040ceb50 0x7f76040c80a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.371 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f76040cbc80 0x7f7604150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.371 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 --2- 172.21.15.47:0/4269410593 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f76040ccea0 0x7f760416a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.371 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 -- 172.21.15.47:0/4269410593 >> 172.21.15.47:0/4269410593 conn(0x7f7604099870 msgr2=0x7f76040c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:06.371 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 -- 172.21.15.47:0/4269410593 shutdown_connections 2023-06-22T17:04:06.371 INFO:teuthology.misc.health.smithi047.stderr:2023-06-22T17:04:06.368+0000 7f760b114700 1 -- 172.21.15.47:0/4269410593 wait complete. 2023-06-22T17:04:06.381 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-06-22T17:04:06.381 INFO:tasks.ceph:Creating RBD pool 2023-06-22T17:04:06.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool create rbd 8 2023-06-22T17:04:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.500+0000 7f31dfa5f700 1 Processor -- start 2023-06-22T17:04:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.500+0000 7f31dfa5f700 1 -- start start 2023-06-22T17:04:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.500+0000 7f31dfa5f700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f31d8153a80 0x7f31d80d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dfa5f700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 0x7f31d815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dfa5f700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d80cedd0 con 0x7f31d80b2010 2023-06-22T17:04:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dfa5f700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d80d14c0 con 0x7f31d8153a80 2023-06-22T17:04:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dfa5f700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80ceac0 con 0x7f31d8153e50 2023-06-22T17:04:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dcffa700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 0x7f31d815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dd7fb700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f31d8153a80 0x7f31d80d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31ddffc700 1 --1- >> v1:172.21.15.202:6789/0 conn(0x7f31d80b2010 0x7f31d815d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.202:6789/0 says I am v1:172.21.15.47:42788/0 (socket says 172.21.15.47:42788) 2023-06-22T17:04:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31ddffc700 1 -- 172.21.15.47:0/807164409 learned_addr learned my addr 172.21.15.47:0/807164409 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dd7fb700 1 -- 172.21.15.47:0/807164409 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 msgr2=0x7f31d815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dd7fb700 1 --2- 172.21.15.47:0/807164409 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 0x7f31d815a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dd7fb700 1 -- 172.21.15.47:0/807164409 >> v1:172.21.15.202:6789/0 conn(0x7f31d80b2010 legacy=0x7f31d815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dd7fb700 1 -- 172.21.15.47:0/807164409 --> v2:172.21.15.202:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d80d0c20 con 0x7f31d8153a80 2023-06-22T17:04:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.501+0000 7f31dcffa700 1 --2- 172.21.15.47:0/807164409 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 0x7f31d815a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T17:04:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31dd7fb700 1 --2- 172.21.15.47:0/807164409 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f31d8153a80 0x7f31d80d0590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009e60 tx=0x7f31c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7895e4058f4d53 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31d7fff700 1 -- 172.21.15.47:0/807164409 <== mon.1 v2:172.21.15.202:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c800e070 con 0x7f31d8153a80 2023-06-22T17:04:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31d7fff700 1 -- 172.21.15.47:0/807164409 <== mon.1 v2:172.21.15.202:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c800bb60 con 0x7f31d8153a80 2023-06-22T17:04:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31d7fff700 1 -- 172.21.15.47:0/807164409 <== mon.1 v2:172.21.15.202:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8012430 con 0x7f31d8153a80 2023-06-22T17:04:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31dfa5f700 1 -- 172.21.15.47:0/807164409 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f31d8153a80 msgr2=0x7f31d80d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31dfa5f700 1 --2- 172.21.15.47:0/807164409 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f31d8153a80 0x7f31d80d0590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009e60 tx=0x7f31c8009780 comp rx=0 tx=0).stop 2023-06-22T17:04:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31dfa5f700 1 -- 172.21.15.47:0/807164409 shutdown_connections 2023-06-22T17:04:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31dfa5f700 1 --2- 172.21.15.47:0/807164409 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f31d8153a80 0x7f31d80d0590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31dfa5f700 1 --2- 172.21.15.47:0/807164409 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 0x7f31d815a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31dfa5f700 1 -- 172.21.15.47:0/807164409 >> 172.21.15.47:0/807164409 conn(0x7f31d8070290 msgr2=0x7f31d8070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31dfa5f700 1 -- 172.21.15.47:0/807164409 shutdown_connections 2023-06-22T17:04:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.502+0000 7f31dfa5f700 1 -- 172.21.15.47:0/807164409 wait complete. 2023-06-22T17:04:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.503+0000 7f31dfa5f700 1 Processor -- start 2023-06-22T17:04:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.504+0000 7f31dfa5f700 1 -- start start 2023-06-22T17:04:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.504+0000 7f31dfa5f700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f31d80b2010 0x7f31d815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.504+0000 7f31dfa5f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d8153a80 0x7f31d814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.504+0000 7f31dfa5f700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 0x7f31d814a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.504+0000 7f31dfa5f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d814c690 con 0x7f31d8153a80 2023-06-22T17:04:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.504+0000 7f31dfa5f700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d815b080 con 0x7f31d80b2010 2023-06-22T17:04:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.504+0000 7f31dfa5f700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d815b360 con 0x7f31d8153e50 2023-06-22T17:04:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dcffa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d8153a80 0x7f31d814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dcffa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d8153a80 0x7f31d814def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33872/0 (socket says 172.21.15.47:33872) 2023-06-22T17:04:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dcffa700 1 -- 172.21.15.47:0/3229685247 learned_addr learned my addr 172.21.15.47:0/3229685247 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dd7fb700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f31d80b2010 0x7f31d815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31ddffc700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 0x7f31d814a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dcffa700 1 -- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 msgr2=0x7f31d814a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dcffa700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 0x7f31d814a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dcffa700 1 -- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f31d80b2010 msgr2=0x7f31d815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dcffa700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f31d80b2010 0x7f31d815aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dcffa700 1 -- 172.21.15.47:0/3229685247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc009420 con 0x7f31d8153a80 2023-06-22T17:04:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dcffa700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d8153a80 0x7f31d814def0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00ec80 tx=0x7f31cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31d57fa700 1 -- 172.21.15.47:0/3229685247 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00c9d0 con 0x7f31d8153a80 2023-06-22T17:04:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31d57fa700 1 -- 172.21.15.47:0/3229685247 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31cc00cb30 con 0x7f31d8153a80 2023-06-22T17:04:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dfa5f700 1 -- 172.21.15.47:0/3229685247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c8009a20 con 0x7f31d8153a80 2023-06-22T17:04:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31d57fa700 1 -- 172.21.15.47:0/3229685247 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00ceb0 con 0x7f31d8153a80 2023-06-22T17:04:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.505+0000 7f31dfa5f700 1 -- 172.21.15.47:0/3229685247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d815cda0 con 0x7f31d8153a80 2023-06-22T17:04:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.506+0000 7f31dfa5f700 1 -- 172.21.15.47:0/3229685247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d804c1b0 con 0x7f31d8153a80 2023-06-22T17:04:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.507+0000 7f31d57fa700 1 -- 172.21.15.47:0/3229685247 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f31cc013680 con 0x7f31d8153a80 2023-06-22T17:04:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.507+0000 7f31d57fa700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f31c4066e90 0x7f31c4069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.507+0000 7f31d57fa700 1 -- 172.21.15.47:0/3229685247 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f31cc0844f0 con 0x7f31d8153a80 2023-06-22T17:04:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.507+0000 7f31dd7fb700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f31c4066e90 0x7f31c4069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.508+0000 7f31dd7fb700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f31c4066e90 0x7f31c4069340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f31c80097c0 tx=0x7f31c8019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.511+0000 7f31d57fa700 1 -- 172.21.15.47:0/3229685247 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f31cc051b70 con 0x7f31d8153a80 2023-06-22T17:04:06.711 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:06.709+0000 7f31dfa5f700 1 -- 172.21.15.47:0/3229685247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f31d814c7d0 con 0x7f31d8153a80 2023-06-22T17:04:07.410 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.408+0000 7f31d57fa700 1 -- 172.21.15.47:0/3229685247 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7f31cc055190 con 0x7f31d8153a80 2023-06-22T17:04:07.413 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.411+0000 7f31dfa5f700 1 -- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f31c4066e90 msgr2=0x7f31c4069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:07.413 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.411+0000 7f31dfa5f700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f31c4066e90 0x7f31c4069340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f31c80097c0 tx=0x7f31c8019000 comp rx=0 tx=0).stop 2023-06-22T17:04:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.411+0000 7f31dfa5f700 1 -- 172.21.15.47:0/3229685247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d8153a80 msgr2=0x7f31d814def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.411+0000 7f31dfa5f700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d8153a80 0x7f31d814def0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00ec80 tx=0x7f31cc0091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.411+0000 7f31dfa5f700 1 -- 172.21.15.47:0/3229685247 shutdown_connections 2023-06-22T17:04:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.411+0000 7f31dfa5f700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f31c4066e90 0x7f31c4069340 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.411+0000 7f31dfa5f700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f31d8153e50 0x7f31d814a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.411+0000 7f31dfa5f700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f31d80b2010 0x7f31d815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.411+0000 7f31dfa5f700 1 --2- 172.21.15.47:0/3229685247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d8153a80 0x7f31d814def0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.412+0000 7f31dfa5f700 1 -- 172.21.15.47:0/3229685247 >> 172.21.15.47:0/3229685247 conn(0x7f31d8070290 msgr2=0x7f31d80b5640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.412+0000 7f31dfa5f700 1 -- 172.21.15.47:0/3229685247 shutdown_connections 2023-06-22T17:04:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.412+0000 7f31dfa5f700 1 -- 172.21.15.47:0/3229685247 wait complete. 2023-06-22T17:04:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'rbd' created 2023-06-22T17:04:07.427 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-06-22T17:04:07.550 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.548+0000 7fa56b8c5700 1 Processor -- start 2023-06-22T17:04:07.550 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.549+0000 7fa56b8c5700 1 -- start start 2023-06-22T17:04:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.549+0000 7fa56b8c5700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fa56415cde0 0x7fa56415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.549+0000 7fa56b8c5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5640ce800 con 0x7fa5640ce360 2023-06-22T17:04:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.549+0000 7fa56b8c5700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5640ceae0 con 0x7fa5640b3600 2023-06-22T17:04:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.549+0000 7fa56b8c5700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5640cc8f0 con 0x7fa56415cde0 2023-06-22T17:04:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.549+0000 7fa568e60700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa5640ce360 0x7fa5640cd940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50794/0 (socket says 172.21.15.47:50794) 2023-06-22T17:04:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.549+0000 7fa568e60700 1 -- 172.21.15.47:0/3441365610 learned_addr learned my addr 172.21.15.47:0/3441365610 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.550+0000 7fa569661700 1 --2- 172.21.15.47:0/3441365610 >> v2:172.21.15.202:3300/0 conn(0x7fa56415cde0 0x7fa56415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.550+0000 7fa563fff700 1 -- 172.21.15.47:0/3441365610 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245072670 0 0) 0x7fa5640ce800 con 0x7fa5640ce360 2023-06-22T17:04:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.550+0000 7fa563fff700 1 -- 172.21.15.47:0/3441365610 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa54c003570 con 0x7fa5640ce360 2023-06-22T17:04:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.550+0000 7fa569661700 1 -- 172.21.15.47:0/3441365610 >> v1:172.21.15.202:6789/0 conn(0x7fa5640b3600 legacy=0x7fa5640cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.550+0000 7fa569661700 1 -- 172.21.15.47:0/3441365610 >> v1:172.21.15.47:6789/0 conn(0x7fa5640ce360 legacy=0x7fa5640cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.550+0000 7fa563fff700 1 -- 172.21.15.47:0/3441365610 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3945775101 0 0) 0x7fa54c003570 con 0x7fa5640ce360 2023-06-22T17:04:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.550+0000 7fa569661700 1 -- 172.21.15.47:0/3441365610 --> v2:172.21.15.202:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5640cedc0 con 0x7fa56415cde0 2023-06-22T17:04:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.550+0000 7fa569661700 1 --2- 172.21.15.47:0/3441365610 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa56415cde0 0x7fa56415d1b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa554009eb0 tx=0x7fa554009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43acbeed75a4052b server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.550+0000 7fa563fff700 1 -- 172.21.15.47:0/3441365610 <== mon.1 v2:172.21.15.202:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55400e070 con 0x7fa56415cde0 2023-06-22T17:04:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.551+0000 7fa563fff700 1 -- 172.21.15.47:0/3441365610 <== mon.1 v2:172.21.15.202:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55400bc30 con 0x7fa56415cde0 2023-06-22T17:04:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.551+0000 7fa563fff700 1 -- 172.21.15.47:0/3441365610 <== mon.1 v2:172.21.15.202:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa554012430 con 0x7fa56415cde0 2023-06-22T17:04:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.551+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3441365610 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa56415cde0 msgr2=0x7fa56415d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.551+0000 7fa56b8c5700 1 --2- 172.21.15.47:0/3441365610 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa56415cde0 0x7fa56415d1b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa554009eb0 tx=0x7fa554009780 comp rx=0 tx=0).stop 2023-06-22T17:04:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.551+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3441365610 shutdown_connections 2023-06-22T17:04:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.551+0000 7fa56b8c5700 1 --2- 172.21.15.47:0/3441365610 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa56415cde0 0x7fa56415d1b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.551+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3441365610 >> 172.21.15.47:0/3441365610 conn(0x7fa564099a40 msgr2=0x7fa564070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.551+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3441365610 shutdown_connections 2023-06-22T17:04:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.551+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3441365610 wait complete. 2023-06-22T17:04:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.552+0000 7fa56b8c5700 1 Processor -- start 2023-06-22T17:04:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.552+0000 7fa56b8c5700 1 -- start start 2023-06-22T17:04:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa56b8c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5640b3600 0x7fa5640b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa56b8c5700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa5640ce360 0x7fa564151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa56b8c5700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa56415cde0 0x7fa564152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa56b8c5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640c92a0 con 0x7fa5640b3600 2023-06-22T17:04:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa56b8c5700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640c7840 con 0x7fa5640ce360 2023-06-22T17:04:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa56b8c5700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa564154910 con 0x7fa56415cde0 2023-06-22T17:04:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa569661700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5640b3600 0x7fa5640b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa569661700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5640b3600 0x7fa5640b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33878/0 (socket says 172.21.15.47:33878) 2023-06-22T17:04:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa569661700 1 -- 172.21.15.47:0/3234546807 learned_addr learned my addr 172.21.15.47:0/3234546807 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa568e60700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa5640ce360 0x7fa564151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.553+0000 7fa569e62700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa56415cde0 0x7fa564152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa569661700 1 -- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa56415cde0 msgr2=0x7fa564152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa569661700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa56415cde0 0x7fa564152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa569661700 1 -- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa5640ce360 msgr2=0x7fa564151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa569661700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa5640ce360 0x7fa564151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa569661700 1 -- 172.21.15.47:0/3234546807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa554009a20 con 0x7fa5640b3600 2023-06-22T17:04:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa569661700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5640b3600 0x7fa5640b3f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa5540099f0 tx=0x7fa554019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa5617fa700 1 -- 172.21.15.47:0/3234546807 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa554012d90 con 0x7fa5640b3600 2023-06-22T17:04:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa5617fa700 1 -- 172.21.15.47:0/3234546807 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa554027470 con 0x7fa5640b3600 2023-06-22T17:04:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3234546807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa564154bf0 con 0x7fa5640b3600 2023-06-22T17:04:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa5617fa700 1 -- 172.21.15.47:0/3234546807 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa554027830 con 0x7fa5640b3600 2023-06-22T17:04:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.554+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3234546807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5641551b0 con 0x7fa5640b3600 2023-06-22T17:04:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.555+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3234546807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56404c1b0 con 0x7fa5640b3600 2023-06-22T17:04:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.555+0000 7fa5617fa700 1 -- 172.21.15.47:0/3234546807 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fa554024070 con 0x7fa5640b3600 2023-06-22T17:04:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.556+0000 7fa5617fa700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa550066e90 0x7fa550069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.556+0000 7fa5617fa700 1 -- 172.21.15.47:0/3234546807 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fa554086850 con 0x7fa5640b3600 2023-06-22T17:04:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.556+0000 7fa568e60700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa550066e90 0x7fa550069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.557+0000 7fa568e60700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa550066e90 0x7fa550069340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa564159190 tx=0x7fa558007ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.561+0000 7fa5617fa700 1 -- 172.21.15.47:0/3234546807 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fa554053d80 con 0x7fa5640b3600 2023-06-22T17:04:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:07.758+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3234546807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa564099ed0 con 0x7fa5640b3600 2023-06-22T17:04:08.412 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.410+0000 7fa5617fa700 1 -- 172.21.15.47:0/3234546807 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa5540573a0 con 0x7fa5640b3600 2023-06-22T17:04:08.415 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.414+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa550066e90 msgr2=0x7fa550069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.416 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.414+0000 7fa56b8c5700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa550066e90 0x7fa550069340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa564159190 tx=0x7fa558007ad0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.416 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.414+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3234546807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5640b3600 msgr2=0x7fa5640b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.416 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.414+0000 7fa56b8c5700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5640b3600 0x7fa5640b3f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa5540099f0 tx=0x7fa554019000 comp rx=0 tx=0).stop 2023-06-22T17:04:08.416 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.414+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3234546807 shutdown_connections 2023-06-22T17:04:08.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.414+0000 7fa56b8c5700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa550066e90 0x7fa550069340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.414+0000 7fa56b8c5700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa56415cde0 0x7fa564152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.414+0000 7fa56b8c5700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5640b3600 0x7fa5640b3f00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.414+0000 7fa56b8c5700 1 --2- 172.21.15.47:0/3234546807 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa5640ce360 0x7fa564151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.414+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3234546807 >> 172.21.15.47:0/3234546807 conn(0x7fa564099a40 msgr2=0x7fa5640b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:08.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.415+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3234546807 shutdown_connections 2023-06-22T17:04:08.417 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.415+0000 7fa56b8c5700 1 -- 172.21.15.47:0/3234546807 wait complete. 2023-06-22T17:04:08.418 INFO:teuthology.orchestra.run.smithi047.stderr:enabled application 'rbd' on pool 'rbd' 2023-06-22T17:04:08.432 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-06-22T17:04:08.433 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-22T17:04:08.433 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-22T17:04:08.435 INFO:tasks.ceph.mds.a:Started 2023-06-22T17:04:08.435 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-22T17:04:08.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-22T17:04:08.437 INFO:tasks.ceph.mds.c:Started 2023-06-22T17:04:08.438 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-22T17:04:08.438 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-22T17:04:08.440 INFO:tasks.ceph.mds.b:Started 2023-06-22T17:04:08.441 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-22T17:04:08.441 DEBUG:teuthology.orchestra.run.smithi202:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-22T17:04:08.443 INFO:tasks.ceph.mds.d:Started 2023-06-22T17:04:08.443 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-06-22T17:04:08.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-22T17:04:08.517 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2023-06-22T17:04:08.518 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2023-06-22T17:04:08.523 INFO:tasks.ceph.mds.b.smithi202.stdout:starting mds.b at 2023-06-22T17:04:08.523 INFO:tasks.ceph.mds.d.smithi202.stdout:starting mds.d at 2023-06-22T17:04:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.574+0000 7fcdafe26700 1 Processor -- start 2023-06-22T17:04:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.575+0000 7fcdafe26700 1 -- start start 2023-06-22T17:04:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.575+0000 7fcdafe26700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda8159800 0x7fcda8068380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.575+0000 7fcdafe26700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda80ccec0 con 0x7fcda8159430 2023-06-22T17:04:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.575+0000 7fcdafe26700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda80cc2b0 con 0x7fcda80cbd60 2023-06-22T17:04:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.575+0000 7fcdafe26700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80b2ea0 con 0x7fcda8159800 2023-06-22T17:04:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.575+0000 7fcdadbc2700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcda8159430 0x7fcda80cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50798/0 (socket says 172.21.15.47:50798) 2023-06-22T17:04:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.575+0000 7fcdadbc2700 1 -- 172.21.15.47:0/461340839 learned_addr learned my addr 172.21.15.47:0/461340839 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdae3c3700 1 --2- 172.21.15.47:0/461340839 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda8159800 0x7fcda8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdacbc0700 1 -- 172.21.15.47:0/461340839 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466154108 0 0) 0x7fcda80ccec0 con 0x7fcda8159430 2023-06-22T17:04:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdacbc0700 1 -- 172.21.15.47:0/461340839 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd94003570 con 0x7fcda8159430 2023-06-22T17:04:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdacbc0700 1 -- 172.21.15.47:0/461340839 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255555359 0 0) 0x7fcda80cc2b0 con 0x7fcda80cbd60 2023-06-22T17:04:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdacbc0700 1 -- 172.21.15.47:0/461340839 --> v1:172.21.15.202:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcda80ccec0 con 0x7fcda80cbd60 2023-06-22T17:04:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdacbc0700 1 -- 172.21.15.47:0/461340839 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3344533011 0 0) 0x7fcd94003570 con 0x7fcda8159430 2023-06-22T17:04:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdae3c3700 1 -- 172.21.15.47:0/461340839 >> v1:172.21.15.202:6789/0 conn(0x7fcda80cbd60 legacy=0x7fcda80b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdae3c3700 1 -- 172.21.15.47:0/461340839 >> v1:172.21.15.47:6789/0 conn(0x7fcda8159430 legacy=0x7fcda80cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdae3c3700 1 -- 172.21.15.47:0/461340839 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda80cc590 con 0x7fcda8159800 2023-06-22T17:04:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdae3c3700 1 --2- 172.21.15.47:0/461340839 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda8159800 0x7fcda8068380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009ec0 tx=0x7fcda0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d88b61bded3488 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdacbc0700 1 -- 172.21.15.47:0/461340839 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000e070 con 0x7fcda8159800 2023-06-22T17:04:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdacbc0700 1 -- 172.21.15.47:0/461340839 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda000b870 con 0x7fcda8159800 2023-06-22T17:04:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.576+0000 7fcdacbc0700 1 -- 172.21.15.47:0/461340839 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4001470 con 0x7fcda8159800 2023-06-22T17:04:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.577+0000 7fcdafe26700 1 -- 172.21.15.47:0/461340839 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda8159800 msgr2=0x7fcda8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.577+0000 7fcdafe26700 1 --2- 172.21.15.47:0/461340839 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda8159800 0x7fcda8068380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009ec0 tx=0x7fcda0009780 comp rx=0 tx=0).stop 2023-06-22T17:04:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.577+0000 7fcdafe26700 1 -- 172.21.15.47:0/461340839 shutdown_connections 2023-06-22T17:04:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.577+0000 7fcdafe26700 1 --2- 172.21.15.47:0/461340839 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda8159800 0x7fcda8068380 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.577+0000 7fcdafe26700 1 -- 172.21.15.47:0/461340839 >> 172.21.15.47:0/461340839 conn(0x7fcda8070290 msgr2=0x7fcda8070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.577+0000 7fcdafe26700 1 -- 172.21.15.47:0/461340839 shutdown_connections 2023-06-22T17:04:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.577+0000 7fcdafe26700 1 -- 172.21.15.47:0/461340839 wait complete. 2023-06-22T17:04:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.578+0000 7fcdafe26700 1 Processor -- start 2023-06-22T17:04:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.578+0000 7fcdafe26700 1 -- start start 2023-06-22T17:04:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdafe26700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda80cbd60 0x7fcda816a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdafe26700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fcda8159430 0x7fcda80c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdafe26700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8159800 0x7fcda80c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdafe26700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80c7370 con 0x7fcda8159800 2023-06-22T17:04:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdafe26700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda815bd40 con 0x7fcda8159430 2023-06-22T17:04:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdafe26700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80c8ac0 con 0x7fcda80cbd60 2023-06-22T17:04:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdae3c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8159800 0x7fcda80c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdae3c3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8159800 0x7fcda80c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33920/0 (socket says 172.21.15.47:33920) 2023-06-22T17:04:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdae3c3700 1 -- 172.21.15.47:0/3256219453 learned_addr learned my addr 172.21.15.47:0/3256219453 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdadbc2700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda80cbd60 0x7fcda816a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdad3c1700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fcda8159430 0x7fcda80c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdae3c3700 1 -- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda80cbd60 msgr2=0x7fcda816a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdae3c3700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda80cbd60 0x7fcda816a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdae3c3700 1 -- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fcda8159430 msgr2=0x7fcda80c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdae3c3700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fcda8159430 0x7fcda80c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdae3c3700 1 -- 172.21.15.47:0/3256219453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda0009a20 con 0x7fcda8159800 2023-06-22T17:04:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.579+0000 7fcdae3c3700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8159800 0x7fcda80c8210 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcda00097c0 tx=0x7fcda0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.580+0000 7fcd9dffb700 1 -- 172.21.15.47:0/3256219453 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0026400 con 0x7fcda8159800 2023-06-22T17:04:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.580+0000 7fcd9dffb700 1 -- 172.21.15.47:0/3256219453 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda00265c0 con 0x7fcda8159800 2023-06-22T17:04:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.580+0000 7fcdafe26700 1 -- 172.21.15.47:0/3256219453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda80c8da0 con 0x7fcda8159800 2023-06-22T17:04:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.580+0000 7fcd9dffb700 1 -- 172.21.15.47:0/3256219453 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0026980 con 0x7fcda8159800 2023-06-22T17:04:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.580+0000 7fcdafe26700 1 -- 172.21.15.47:0/3256219453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda80c9360 con 0x7fcda8159800 2023-06-22T17:04:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.581+0000 7fcdafe26700 1 -- 172.21.15.47:0/3256219453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda815bb10 con 0x7fcda8159800 2023-06-22T17:04:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.581+0000 7fcd9dffb700 1 -- 172.21.15.47:0/3256219453 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fcda0019030 con 0x7fcda8159800 2023-06-22T17:04:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.582+0000 7fcd9dffb700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcd8c066e90 0x7fcd8c069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.582+0000 7fcd9dffb700 1 -- 172.21.15.47:0/3256219453 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fcda0085f60 con 0x7fcda8159800 2023-06-22T17:04:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.582+0000 7fcdadbc2700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcd8c066e90 0x7fcd8c069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.582+0000 7fcdadbc2700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcd8c066e90 0x7fcd8c069340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcda40011e0 tx=0x7fcda4000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.586+0000 7fcd9dffb700 1 -- 172.21.15.47:0/3256219453 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fcda815bb10 con 0x7fcda8159800 2023-06-22T17:04:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.781+0000 7fcdafe26700 1 -- 172.21.15.47:0/3256219453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcda815bb10 con 0x7fcda8159800 2023-06-22T17:04:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.782+0000 7fcd9dffb700 1 -- 172.21.15.47:0/3256219453 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13660 (secure 0 0 0) 0x7fcda815bb10 con 0x7fcda8159800 2023-06-22T17:04:08.784 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:04:08.784 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"fsid":"61e931aa-8dfe-4361-8a29-1d30432b403b","created":"2023-06-22T17:03:48.391484+0000","modified":"2023-06-22T17:04:08.408636+0000","last_up_change":"2023-06-22T17:03:59.369744+0000","last_in_change":"2023-06-22T17:03:50.761615+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-22T17:04:00.277500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-22T17:04:06.740390+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"191ca9cf-dc97-4325-b63b-a3e6a66b71b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":87966},{"type":"v1","addr":"172.21.15.47:6809","nonce":87966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":87966},{"type":"v1","addr":"172.21.15.47:6811","nonce":87966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":87966},{"type":"v1","addr":"172.21.15.47:6815","nonce":87966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":87966},{"type":"v1","addr":"172.21.15.47:6813","nonce":87966}]},"public_addr":"172.21.15.47:6809/87966","cluster_addr":"172.21.15.47:6811/87966","heartbeat_back_addr":"172.21.15.47:6815/87966","heartbeat_front_addr":"172.21.15.47:6813/87966","state":["exists","up"]},{"osd":1,"uuid":"d426e8cd-b1e6-40a3-9e65-cf52fac95a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":87978},{"type":"v1","addr":"172.21.15.47:6825","nonce":87978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":87978},{"type":"v1","addr":"172.21.15.47:6827","nonce":87978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":87978},{"type":"v1","addr":"172.21.15.47:6831","nonce":87978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":87978},{"type":"v1","addr":"172.21.15.47:6829","nonce":87978}]},"public_addr":"172.21.15.47:6825/87978","cluster_addr":"172.21.15.47:6827/87978","heartbeat_back_addr":"172.21.15.47:6831/87978","heartbeat_front_addr":"172.21.15.47:6829/87978","state":["exists","up"]},{"osd":2,"uuid":"487a8fca-1da6-4cb3-a73c-abc6e9e2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":87967},{"type":"v1","addr":"172.21.15.47:6801","nonce":87967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":87967},{"type":"v1","addr":"172.21.15.47:6803","nonce":87967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":87967},{"type":"v1","addr":"172.21.15.47:6807","nonce":87967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":87967},{"type":"v1","addr":"172.21.15.47:6805","nonce":87967}]},"public_addr":"172.21.15.47:6801/87967","cluster_addr":"172.21.15.47:6803/87967","heartbeat_back_addr":"172.21.15.47:6807/87967","heartbeat_front_addr":"172.21.15.47:6805/87967","state":["exists","up"]},{"osd":3,"uuid":"7089499f-dc69-4d6b-a6c3-5b30404ab1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":87968},{"type":"v1","addr":"172.21.15.47:6817","nonce":87968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":87968},{"type":"v1","addr":"172.21.15.47:6819","nonce":87968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":87968},{"type":"v1","addr":"172.21.15.47:6823","nonce":87968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":87968},{"type":"v1","addr":"172.21.15.47:6821","nonce":87968}]},"public_addr":"172.21.15.47:6817/87968","cluster_addr":"172.21.15.47:6819/87968","heartbeat_back_addr":"172.21.15.47:6823/87968","heartbeat_front_addr":"172.21.15.47:6821/87968","state":["exists","up"]},{"osd":4,"uuid":"f8b34dee-7945-42fd-9e58-98e5fe196b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6800","nonce":88201},{"type":"v1","addr":"172.21.15.202:6801","nonce":88201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6802","nonce":88201},{"type":"v1","addr":"172.21.15.202:6803","nonce":88201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6806","nonce":88201},{"type":"v1","addr":"172.21.15.202:6807","nonce":88201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6804","nonce":88201},{"type":"v1","addr":"172.21.15.202:6805","nonce":88201}]},"public_addr":"172.21.15.202:6801/88201","cluster_addr":"172.21.15.202:6803/88201","heartbeat_back_addr":"172.21.15.202:6807/88201","heartbeat_front_addr":"172.21.15.202:6805/88201","state":["exists","up"]},{"osd":5,"uuid":"9583b379-49e1-4b6c-adf5-6f19322cc76c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6808","nonce":88199},{"type":"v1","addr":"172.21.15.202:6809","nonce":88199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6810","nonce":88199},{"type":"v1","addr":"172.21.15.202:6812","nonce":88199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6818","nonce":88199},{"type":"v1","addr":"172.21.15.202:6820","nonce":88199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6814","nonce":88199},{"type":"v1","addr":"172.21.15.202:6816","nonce":88199}]},"public_addr":"172.21.15.202:6809/88199","cluster_addr":"172.21.15.202:6812/88199","heartbeat_back_addr":"172.21.15.202:6820/88199","heartbeat_front_addr":"172.21.15.202:6816/88199","state":["exists","up"]},{"osd":6,"uuid":"d11633b5-5a36-441b-93c7-7c0b434f0dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6811","nonce":88200},{"type":"v1","addr":"172.21.15.202:6813","nonce":88200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6815","nonce":88200},{"type":"v1","addr":"172.21.15.202:6817","nonce":88200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6822","nonce":88200},{"type":"v1","addr":"172.21.15.202:6823","nonce":88200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6819","nonce":88200},{"type":"v1","addr":"172.21.15.202:6821","nonce":88200}]},"public_addr":"172.21.15.202:6813/88200","cluster_addr":"172.21.15.202:6817/88200","heartbeat_back_addr":"172.21.15.202:6823/88200","heartbeat_front_addr":"172.21.15.202:6821/88200","state":["exists","up"]},{"osd":7,"uuid":"edd9bf42-9566-4d07-902b-08cf16cb42a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6824","nonce":88230},{"type":"v1","addr":"172.21.15.202:6825","nonce":88230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6826","nonce":88230},{"type":"v1","addr":"172.21.15.202:6827","nonce":88230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6830","nonce":88230},{"type":"v1","addr":"172.21.15.202:6831","nonce":88230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6828","nonce":88230},{"type":"v1","addr":"172.21.15.202:6829","nonce":88230}]},"public_addr":"172.21.15.202:6825/88230","cluster_addr":"172.21.15.202:6827/88230","heartbeat_back_addr":"172.21.15.202:6831/88230","heartbeat_front_addr":"172.21.15.202:6829/88230","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.975830+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.942848+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.958568+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.972986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.946508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.938793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.960236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.931356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T17:04:08.786 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 -- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcd8c066e90 msgr2=0x7fcd8c069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.786 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcd8c066e90 0x7fcd8c069340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcda40011e0 tx=0x7fcda4000bc0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.786 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 -- 172.21.15.47:0/3256219453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8159800 msgr2=0x7fcda80c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.786 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8159800 0x7fcda80c8210 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcda00097c0 tx=0x7fcda0017000 comp rx=0 tx=0).stop 2023-06-22T17:04:08.786 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 -- 172.21.15.47:0/3256219453 shutdown_connections 2023-06-22T17:04:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcd8c066e90 0x7fcd8c069340 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8159800 0x7fcda80c8210 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcda80cbd60 0x7fcda816a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 --2- 172.21.15.47:0/3256219453 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fcda8159430 0x7fcda80c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 -- 172.21.15.47:0/3256219453 >> 172.21.15.47:0/3256219453 conn(0x7fcda8070290 msgr2=0x7fcda80cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 -- 172.21.15.47:0/3256219453 shutdown_connections 2023-06-22T17:04:08.787 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.784+0000 7fcdafe26700 1 -- 172.21.15.47:0/3256219453 wait complete. 2023-06-22T17:04:08.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-22T17:04:00.277500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-22T17:04:06.740390+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-06-22T17:04:08.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-22T17:04:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.920+0000 7fa7321da700 1 Processor -- start 2023-06-22T17:04:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.920+0000 7fa7321da700 1 -- start start 2023-06-22T17:04:08.924 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.921+0000 7fa7321da700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa72c0b03e0 0x7fa72c068e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:08.924 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.921+0000 7fa7321da700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c0ae370 con 0x7fa72c0a8a00 2023-06-22T17:04:08.925 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.921+0000 7fa7321da700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c0ae610 con 0x7fa72c0cddc0 2023-06-22T17:04:08.925 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.921+0000 7fa7321da700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c0ce360 con 0x7fa72c0b03e0 2023-06-22T17:04:08.925 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.921+0000 7fa72bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa72c0b03e0 0x7fa72c068e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:08.925 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.921+0000 7fa72b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa72c0a8a00 0x7fa72c0a8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57012/0 (socket says 172.21.15.47:57012) 2023-06-22T17:04:08.925 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.921+0000 7fa72bfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa72c0b03e0 0x7fa72c068e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51256/0 (socket says 172.21.15.47:51256) 2023-06-22T17:04:08.925 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.921+0000 7fa72b7fe700 1 -- 172.21.15.47:0/3228604543 learned_addr learned my addr 172.21.15.47:0/3228604543 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:08.925 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.922+0000 7fa72bfff700 1 -- 172.21.15.47:0/3228604543 >> v1:172.21.15.202:6789/0 conn(0x7fa72c0cddc0 legacy=0x7fa72c068730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.925 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.922+0000 7fa72bfff700 1 -- 172.21.15.47:0/3228604543 >> v1:172.21.15.47:6789/0 conn(0x7fa72c0a8a00 legacy=0x7fa72c0a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.922+0000 7fa72bfff700 1 -- 172.21.15.47:0/3228604543 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0ae8f0 con 0x7fa72c0b03e0 2023-06-22T17:04:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.922+0000 7fa72a7fc700 1 -- 172.21.15.47:0/3228604543 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206216906 0 0) 0x7fa72c0ae370 con 0x7fa72c0a8a00 2023-06-22T17:04:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.922+0000 7fa72bfff700 1 --2- 172.21.15.47:0/3228604543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0b03e0 0x7fa72c068e40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa724009e60 tx=0x7fa724009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a28735662b84fa4 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.922+0000 7fa72a7fc700 1 -- 172.21.15.47:0/3228604543 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72400e070 con 0x7fa72c0b03e0 2023-06-22T17:04:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.922+0000 7fa72a7fc700 1 -- 172.21.15.47:0/3228604543 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72400bcc0 con 0x7fa72c0b03e0 2023-06-22T17:04:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.922+0000 7fa72a7fc700 1 -- 172.21.15.47:0/3228604543 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724012680 con 0x7fa72c0b03e0 2023-06-22T17:04:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.922+0000 7fa7321da700 1 -- 172.21.15.47:0/3228604543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0b03e0 msgr2=0x7fa72c068e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.922+0000 7fa7321da700 1 --2- 172.21.15.47:0/3228604543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0b03e0 0x7fa72c068e40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa724009e60 tx=0x7fa724009780 comp rx=0 tx=0).stop 2023-06-22T17:04:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.923+0000 7fa7321da700 1 -- 172.21.15.47:0/3228604543 shutdown_connections 2023-06-22T17:04:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.923+0000 7fa7321da700 1 --2- 172.21.15.47:0/3228604543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0b03e0 0x7fa72c068e40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.923+0000 7fa7321da700 1 -- 172.21.15.47:0/3228604543 >> 172.21.15.47:0/3228604543 conn(0x7fa72c067bf0 msgr2=0x7fa72c0ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.923+0000 7fa7321da700 1 -- 172.21.15.47:0/3228604543 shutdown_connections 2023-06-22T17:04:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.923+0000 7fa7321da700 1 -- 172.21.15.47:0/3228604543 wait complete. 2023-06-22T17:04:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.924+0000 7fa7321da700 1 Processor -- start 2023-06-22T17:04:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.924+0000 7fa7321da700 1 -- start start 2023-06-22T17:04:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7321da700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0a8a00 0x7fa72c158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7321da700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa72c0b03e0 0x7fa72c0990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7321da700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa72c0cddc0 0x7fa72c06be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7321da700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0c9d70 con 0x7fa72c0a8a00 2023-06-22T17:04:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7321da700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c098c50 con 0x7fa72c0cddc0 2023-06-22T17:04:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7321da700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c09ab90 con 0x7fa72c0b03e0 2023-06-22T17:04:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0a8a00 0x7fa72c158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0a8a00 0x7fa72c158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51270/0 (socket says 172.21.15.47:51270) 2023-06-22T17:04:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72b7fe700 1 -- 172.21.15.47:0/2000255870 learned_addr learned my addr 172.21.15.47:0/2000255870 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72affd700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa72c0b03e0 0x7fa72c0990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72bfff700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa72c0cddc0 0x7fa72c06be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72b7fe700 1 -- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa72c0b03e0 msgr2=0x7fa72c0990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72b7fe700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa72c0b03e0 0x7fa72c0990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72b7fe700 1 -- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa72c0cddc0 msgr2=0x7fa72c06be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72b7fe700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa72c0cddc0 0x7fa72c06be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72b7fe700 1 -- 172.21.15.47:0/2000255870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724009a20 con 0x7fa72c0a8a00 2023-06-22T17:04:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72bfff700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa72c0cddc0 0x7fa72c06be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T17:04:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa72b7fe700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0a8a00 0x7fa72c158890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00c590 tx=0x7fa71c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7137fe700 1 -- 172.21.15.47:0/2000255870 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0072e0 con 0x7fa72c0a8a00 2023-06-22T17:04:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7137fe700 1 -- 172.21.15.47:0/2000255870 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c007480 con 0x7fa72c0a8a00 2023-06-22T17:04:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7137fe700 1 -- 172.21.15.47:0/2000255870 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0057f0 con 0x7fa72c0a8a00 2023-06-22T17:04:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7321da700 1 -- 172.21.15.47:0/2000255870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c098320 con 0x7fa72c0a8a00 2023-06-22T17:04:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.925+0000 7fa7321da700 1 -- 172.21.15.47:0/2000255870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c0988e0 con 0x7fa72c0a8a00 2023-06-22T17:04:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.926+0000 7fa7321da700 1 -- 172.21.15.47:0/2000255870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c0cc3f0 con 0x7fa72c0a8a00 2023-06-22T17:04:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.927+0000 7fa7137fe700 1 -- 172.21.15.47:0/2000255870 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fa71c005bc0 con 0x7fa72c0a8a00 2023-06-22T17:04:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.927+0000 7fa7137fe700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa718066e90 0x7fa718069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.927+0000 7fa7137fe700 1 -- 172.21.15.47:0/2000255870 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa71c0835d0 con 0x7fa72c0a8a00 2023-06-22T17:04:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.927+0000 7fa72affd700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa718066e90 0x7fa718069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:08.933 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.927+0000 7fa72affd700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa718066e90 0x7fa718069340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0b42b0 tx=0x7fa720007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:08.934 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:08.931+0000 7fa7137fe700 1 -- 172.21.15.47:0/2000255870 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fa71c04b590 con 0x7fa72c0a8a00 2023-06-22T17:04:09.125 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.124+0000 7fa7321da700 1 -- 172.21.15.47:0/2000255870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa72c04c1b0 con 0x7fa72c0a8a00 2023-06-22T17:04:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.124+0000 7fa7137fe700 1 -- 172.21.15.47:0/2000255870 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa71c050ba0 con 0x7fa72c0a8a00 2023-06-22T17:04:09.126 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2023-06-22T17:04:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 -- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa718066e90 msgr2=0x7fa718069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa718066e90 0x7fa718069340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0b42b0 tx=0x7fa720007ec0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 -- 172.21.15.47:0/2000255870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0a8a00 msgr2=0x7fa72c158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0a8a00 0x7fa72c158890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00c590 tx=0x7fa71c007e80 comp rx=0 tx=0).stop 2023-06-22T17:04:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 -- 172.21.15.47:0/2000255870 shutdown_connections 2023-06-22T17:04:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fa718066e90 0x7fa718069340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.129 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fa72c0cddc0 0x7fa72c06be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.129 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa72c0a8a00 0x7fa72c158890 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.129 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 --2- 172.21.15.47:0/2000255870 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fa72c0b03e0 0x7fa72c0990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.129 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 -- 172.21.15.47:0/2000255870 >> 172.21.15.47:0/2000255870 conn(0x7fa72c067bf0 msgr2=0x7fa72c15b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:09.129 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 -- 172.21.15.47:0/2000255870 shutdown_connections 2023-06-22T17:04:09.129 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.126+0000 7fa7321da700 1 -- 172.21.15.47:0/2000255870 wait complete. 2023-06-22T17:04:09.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-22T17:04:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.260+0000 7fd1a11c7700 1 Processor -- start 2023-06-22T17:04:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.260+0000 7fd1a11c7700 1 -- start start 2023-06-22T17:04:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.260+0000 7fd1a11c7700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fd19c153a80 0x7fd19c0d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.260+0000 7fd1a11c7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd19c0b2010 0x7fd19c0b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd1a11c7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c0cedd0 con 0x7fd19c153e50 2023-06-22T17:04:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd1a11c7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19c0d14c0 con 0x7fd19c0b2010 2023-06-22T17:04:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd1a11c7700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19c0b29d0 con 0x7fd19c153a80 2023-06-22T17:04:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd19b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd19c0b2010 0x7fd19c0b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd19a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd19c153e50 0x7fd19c15a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57024/0 (socket says 172.21.15.47:57024) 2023-06-22T17:04:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd19a59c700 1 -- 172.21.15.47:0/2082226738 learned_addr learned my addr 172.21.15.47:0/2082226738 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd19ad9d700 1 --2- 172.21.15.47:0/2082226738 >> v2:172.21.15.202:3300/0 conn(0x7fd19c153a80 0x7fd19c0d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd199d9b700 1 -- 172.21.15.47:0/2082226738 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422082316 0 0) 0x7fd19c0cedd0 con 0x7fd19c153e50 2023-06-22T17:04:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd199d9b700 1 -- 172.21.15.47:0/2082226738 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd188003570 con 0x7fd19c153e50 2023-06-22T17:04:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd19b59e700 1 -- 172.21.15.47:0/2082226738 >> v2:172.21.15.202:3300/0 conn(0x7fd19c153a80 msgr2=0x7fd19c0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd19b59e700 1 --2- 172.21.15.47:0/2082226738 >> v2:172.21.15.202:3300/0 conn(0x7fd19c153a80 0x7fd19c0d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd19b59e700 1 -- 172.21.15.47:0/2082226738 >> v1:172.21.15.47:6789/0 conn(0x7fd19c153e50 legacy=0x7fd19c15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd19b59e700 1 -- 172.21.15.47:0/2082226738 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c15aed0 con 0x7fd19c0b2010 2023-06-22T17:04:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd19b59e700 1 --2- 172.21.15.47:0/2082226738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c0b2010 0x7fd19c0b2490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0b3100 tx=0x7fd194009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7bbd9d83d03c1f server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd199d9b700 1 -- 172.21.15.47:0/2082226738 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19400e070 con 0x7fd19c0b2010 2023-06-22T17:04:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd199d9b700 1 -- 172.21.15.47:0/2082226738 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19400bb20 con 0x7fd19c0b2010 2023-06-22T17:04:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.261+0000 7fd199d9b700 1 -- 172.21.15.47:0/2082226738 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194012430 con 0x7fd19c0b2010 2023-06-22T17:04:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.262+0000 7fd1a11c7700 1 -- 172.21.15.47:0/2082226738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c0b2010 msgr2=0x7fd19c0b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.262+0000 7fd1a11c7700 1 --2- 172.21.15.47:0/2082226738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c0b2010 0x7fd19c0b2490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0b3100 tx=0x7fd194009780 comp rx=0 tx=0).stop 2023-06-22T17:04:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.262+0000 7fd1a11c7700 1 -- 172.21.15.47:0/2082226738 shutdown_connections 2023-06-22T17:04:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.262+0000 7fd1a11c7700 1 --2- 172.21.15.47:0/2082226738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c0b2010 0x7fd19c0b2490 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.262+0000 7fd1a11c7700 1 --2- 172.21.15.47:0/2082226738 >> v2:172.21.15.202:3300/0 conn(0x7fd19c153a80 0x7fd19c0d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.262+0000 7fd1a11c7700 1 -- 172.21.15.47:0/2082226738 >> 172.21.15.47:0/2082226738 conn(0x7fd19c070290 msgr2=0x7fd19c070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.262+0000 7fd1a11c7700 1 -- 172.21.15.47:0/2082226738 shutdown_connections 2023-06-22T17:04:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.262+0000 7fd1a11c7700 1 -- 172.21.15.47:0/2082226738 wait complete. 2023-06-22T17:04:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.263+0000 7fd1a11c7700 1 Processor -- start 2023-06-22T17:04:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.263+0000 7fd1a11c7700 1 -- start start 2023-06-22T17:04:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd1a11c7700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fd19c0b2010 0x7fd19c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd1a11c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c153a80 0x7fd19c151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd1a11c7700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fd19c153e50 0x7fd19c152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd1a11c7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c157590 con 0x7fd19c153a80 2023-06-22T17:04:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd1a11c7700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c169810 con 0x7fd19c153e50 2023-06-22T17:04:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd1a11c7700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c152880 con 0x7fd19c0b2010 2023-06-22T17:04:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c153a80 0x7fd19c151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c153a80 0x7fd19c151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51296/0 (socket says 172.21.15.47:51296) 2023-06-22T17:04:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19a59c700 1 -- 172.21.15.47:0/1585685594 learned_addr learned my addr 172.21.15.47:0/1585685594 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19b59e700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fd19c153e50 0x7fd19c152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19ad9d700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fd19c0b2010 0x7fd19c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19a59c700 1 -- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fd19c0b2010 msgr2=0x7fd19c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19a59c700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fd19c0b2010 0x7fd19c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19a59c700 1 -- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fd19c153e50 msgr2=0x7fd19c152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19a59c700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fd19c153e50 0x7fd19c152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19a59c700 1 -- 172.21.15.47:0/1585685594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd194009a20 con 0x7fd19c153a80 2023-06-22T17:04:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19b59e700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fd19c153e50 0x7fd19c152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T17:04:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.264+0000 7fd19a59c700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c153a80 0x7fd19c151e00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c7c0 tx=0x7fd184007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.265+0000 7fd192ffd700 1 -- 172.21.15.47:0/1585685594 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184005b30 con 0x7fd19c153a80 2023-06-22T17:04:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.265+0000 7fd192ffd700 1 -- 172.21.15.47:0/1585685594 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184005c90 con 0x7fd19c153a80 2023-06-22T17:04:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.265+0000 7fd192ffd700 1 -- 172.21.15.47:0/1585685594 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184007320 con 0x7fd19c153a80 2023-06-22T17:04:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.265+0000 7fd1a11c7700 1 -- 172.21.15.47:0/1585685594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c156a10 con 0x7fd19c153a80 2023-06-22T17:04:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.265+0000 7fd1a11c7700 1 -- 172.21.15.47:0/1585685594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c1572b0 con 0x7fd19c153a80 2023-06-22T17:04:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.266+0000 7fd1a11c7700 1 -- 172.21.15.47:0/1585685594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c04c1b0 con 0x7fd19c153a80 2023-06-22T17:04:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.266+0000 7fd192ffd700 1 -- 172.21.15.47:0/1585685594 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fd184007690 con 0x7fd19c153a80 2023-06-22T17:04:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.266+0000 7fd192ffd700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fd17c066e90 0x7fd17c069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.266+0000 7fd192ffd700 1 -- 172.21.15.47:0/1585685594 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd184083870 con 0x7fd19c153a80 2023-06-22T17:04:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.267+0000 7fd19ad9d700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fd17c066e90 0x7fd17c069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.267+0000 7fd19ad9d700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fd17c066e90 0x7fd17c069340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd18c006fd0 tx=0x7fd18c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.271+0000 7fd192ffd700 1 -- 172.21.15.47:0/1585685594 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fd184050cf0 con 0x7fd19c153a80 2023-06-22T17:04:09.466 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.464+0000 7fd1a11c7700 1 -- 172.21.15.47:0/1585685594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd19c0b6420 con 0x7fd19c153a80 2023-06-22T17:04:09.466 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.464+0000 7fd192ffd700 1 -- 172.21.15.47:0/1585685594 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd184017020 con 0x7fd19c153a80 2023-06-22T17:04:09.466 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2023-06-22T17:04:09.468 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 -- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fd17c066e90 msgr2=0x7fd17c069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.468 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fd17c066e90 0x7fd17c069340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd18c006fd0 tx=0x7fd18c008040 comp rx=0 tx=0).stop 2023-06-22T17:04:09.468 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 -- 172.21.15.47:0/1585685594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c153a80 msgr2=0x7fd19c151e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c153a80 0x7fd19c151e00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c7c0 tx=0x7fd184007e00 comp rx=0 tx=0).stop 2023-06-22T17:04:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 -- 172.21.15.47:0/1585685594 shutdown_connections 2023-06-22T17:04:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fd17c066e90 0x7fd17c069340 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fd19c153e50 0x7fd19c152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fd19c0b2010 0x7fd19c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 --2- 172.21.15.47:0/1585685594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c153a80 0x7fd19c151e00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 -- 172.21.15.47:0/1585685594 >> 172.21.15.47:0/1585685594 conn(0x7fd19c070290 msgr2=0x7fd19c15b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 -- 172.21.15.47:0/1585685594 shutdown_connections 2023-06-22T17:04:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.466+0000 7fd1a11c7700 1 -- 172.21.15.47:0/1585685594 wait complete. 2023-06-22T17:04:09.480 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T17:04:09.601 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.599+0000 7f0860c19700 1 Processor -- start 2023-06-22T17:04:09.601 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.599+0000 7f0860c19700 1 -- start start 2023-06-22T17:04:09.602 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f0860c19700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f085c0c93a0 0x7f085c0b1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.602 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f0860c19700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f085c0c4a60 0x7f085c0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.602 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f0860c19700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f085c0c3ec0 con 0x7f085c0c9770 2023-06-22T17:04:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f0860c19700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f085c0c3010 con 0x7f085c0c93a0 2023-06-22T17:04:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f0860c19700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0c32b0 con 0x7f085c0c4a60 2023-06-22T17:04:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f085b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f085c0c93a0 0x7f085c0b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f085b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f085c0c93a0 0x7f085c0b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51300/0 (socket says 172.21.15.47:51300) 2023-06-22T17:04:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f085b7fe700 1 -- 172.21.15.47:0/1545552559 learned_addr learned my addr 172.21.15.47:0/1545552559 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f085b7fe700 1 -- 172.21.15.47:0/1545552559 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f085c0c4a60 msgr2=0x7f085c0cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:09.603 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f085b7fe700 1 --2- 172.21.15.47:0/1545552559 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f085c0c4a60 0x7f085c0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f085b7fe700 1 -- 172.21.15.47:0/1545552559 >> v1:172.21.15.202:6789/0 conn(0x7f085c0c9770 legacy=0x7f085c0c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f085b7fe700 1 -- 172.21.15.47:0/1545552559 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c0c4ee0 con 0x7f085c0c93a0 2023-06-22T17:04:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.600+0000 7f085b7fe700 1 --2- 172.21.15.47:0/1545552559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c93a0 0x7f085c0b1030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f084c009990 tx=0x7f084c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90aa07d15328d6fe server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f085a7fc700 1 -- 172.21.15.47:0/1545552559 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c005c10 con 0x7f085c0c93a0 2023-06-22T17:04:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f085a7fc700 1 -- 172.21.15.47:0/1545552559 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c005d70 con 0x7f085c0c93a0 2023-06-22T17:04:09.604 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f085a7fc700 1 -- 172.21.15.47:0/1545552559 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c00b8d0 con 0x7f085c0c93a0 2023-06-22T17:04:09.606 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f0860c19700 1 -- 172.21.15.47:0/1545552559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c93a0 msgr2=0x7f085c0b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.607 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f0860c19700 1 --2- 172.21.15.47:0/1545552559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c93a0 0x7f085c0b1030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f084c009990 tx=0x7f084c009220 comp rx=0 tx=0).stop 2023-06-22T17:04:09.607 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f0860c19700 1 -- 172.21.15.47:0/1545552559 shutdown_connections 2023-06-22T17:04:09.607 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f0860c19700 1 --2- 172.21.15.47:0/1545552559 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f085c0c4a60 0x7f085c0cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.607 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f0860c19700 1 --2- 172.21.15.47:0/1545552559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c93a0 0x7f085c0b1030 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.607 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f0860c19700 1 -- 172.21.15.47:0/1545552559 >> 172.21.15.47:0/1545552559 conn(0x7f085c067bf0 msgr2=0x7f085c0ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:09.607 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f0860c19700 1 -- 172.21.15.47:0/1545552559 shutdown_connections 2023-06-22T17:04:09.607 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.601+0000 7f0860c19700 1 -- 172.21.15.47:0/1545552559 wait complete. 2023-06-22T17:04:09.607 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f0860c19700 1 Processor -- start 2023-06-22T17:04:09.608 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f0860c19700 1 -- start start 2023-06-22T17:04:09.608 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f0860c19700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c4a60 0x7f085c142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.608 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f0860c19700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f085c0c93a0 0x7f085c0c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.608 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f0860c19700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f085c0c9770 0x7f085c0c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.608 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f0860c19700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c143210 con 0x7f085c0c4a60 2023-06-22T17:04:09.608 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f0860c19700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0c5d70 con 0x7f085c0c9770 2023-06-22T17:04:09.608 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f0860c19700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0c6050 con 0x7f085c0c93a0 2023-06-22T17:04:09.609 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f085b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c4a60 0x7f085c142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.609 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f085b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c4a60 0x7f085c142cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51308/0 (socket says 172.21.15.47:51308) 2023-06-22T17:04:09.609 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.603+0000 7f085b7fe700 1 -- 172.21.15.47:0/1624571131 learned_addr learned my addr 172.21.15.47:0/1624571131 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:09.609 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f085bfff700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f085c0c9770 0x7f085c0c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.609 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f085affd700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f085c0c93a0 0x7f085c0c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.609 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f085b7fe700 1 -- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f085c0c93a0 msgr2=0x7f085c0c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.609 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f085b7fe700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f085c0c93a0 0x7f085c0c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.609 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f085b7fe700 1 -- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f085c0c9770 msgr2=0x7f085c0c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.610 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f085b7fe700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f085c0c9770 0x7f085c0c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.610 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f085b7fe700 1 -- 172.21.15.47:0/1624571131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c009480 con 0x7f085c0c4a60 2023-06-22T17:04:09.610 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f085b7fe700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c4a60 0x7f085c142cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f084c009260 tx=0x7f084c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:09.610 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f08437fe700 1 -- 172.21.15.47:0/1624571131 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c000d10 con 0x7f085c0c4a60 2023-06-22T17:04:09.610 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f08437fe700 1 -- 172.21.15.47:0/1624571131 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c011c80 con 0x7f085c0c4a60 2023-06-22T17:04:09.610 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f0860c19700 1 -- 172.21.15.47:0/1624571131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085c09ad60 con 0x7f085c0c4a60 2023-06-22T17:04:09.610 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f08437fe700 1 -- 172.21.15.47:0/1624571131 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c01d430 con 0x7f085c0c4a60 2023-06-22T17:04:09.610 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.604+0000 7f0860c19700 1 -- 172.21.15.47:0/1624571131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c09aa30 con 0x7f085c0c4a60 2023-06-22T17:04:09.611 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.605+0000 7f0860c19700 1 -- 172.21.15.47:0/1624571131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085c09b200 con 0x7f085c0c4a60 2023-06-22T17:04:09.611 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.605+0000 7f08437fe700 1 -- 172.21.15.47:0/1624571131 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f084c01d7a0 con 0x7f085c0c4a60 2023-06-22T17:04:09.611 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.606+0000 7f08437fe700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f0848066e90 0x7f0848069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.612 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.606+0000 7f08437fe700 1 -- 172.21.15.47:0/1624571131 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f084c086060 con 0x7f085c0c4a60 2023-06-22T17:04:09.612 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.606+0000 7f085affd700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f0848066e90 0x7f0848069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.612 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.606+0000 7f085affd700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f0848066e90 0x7f0848069340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0850006fd0 tx=0x7f0850008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:09.612 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.610+0000 7f08437fe700 1 -- 172.21.15.47:0/1624571131 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f084c053580 con 0x7f085c0c4a60 2023-06-22T17:04:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.853+0000 7f0860c19700 1 -- 172.21.15.47:0/1624571131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f085c0c8680 con 0x7f085c0c4a60 2023-06-22T17:04:09.855 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.853+0000 7f08437fe700 1 -- 172.21.15.47:0/1624571131 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f084c056ba0 con 0x7f085c0c4a60 2023-06-22T17:04:09.855 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:04:09.855 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2765490083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6833","nonce":2765490083},{"type":"v1","addr":"172.21.15.47:6835","nonce":2765490083}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.202:6836/4145074458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6834","nonce":4145074458},{"type":"v1","addr":"172.21.15.202:6836","nonce":4145074458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.202:6837/2182722603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6835","nonce":2182722603},{"type":"v1","addr":"172.21.15.202:6837","nonce":2182722603}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6834/2969740359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2969740359},{"type":"v1","addr":"172.21.15.47:6834","nonce":2969740359}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-22T17:04:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 -- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f0848066e90 msgr2=0x7f0848069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f0848066e90 0x7f0848069340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0850006fd0 tx=0x7f0850008040 comp rx=0 tx=0).stop 2023-06-22T17:04:09.857 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 -- 172.21.15.47:0/1624571131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c4a60 msgr2=0x7f085c142cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.858 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c4a60 0x7f085c142cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f084c009260 tx=0x7f084c015000 comp rx=0 tx=0).stop 2023-06-22T17:04:09.858 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 -- 172.21.15.47:0/1624571131 shutdown_connections 2023-06-22T17:04:09.858 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f0848066e90 0x7f0848069340 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.858 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f085c0c9770 0x7f085c0c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.858 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f085c0c4a60 0x7f085c142cd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.858 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 --2- 172.21.15.47:0/1624571131 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f085c0c93a0 0x7f085c0c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.859 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 -- 172.21.15.47:0/1624571131 >> 172.21.15.47:0/1624571131 conn(0x7f085c067bf0 msgr2=0x7f085c153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:09.859 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 -- 172.21.15.47:0/1624571131 shutdown_connections 2023-06-22T17:04:09.859 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.855+0000 7f0860c19700 1 -- 172.21.15.47:0/1624571131 wait complete. 2023-06-22T17:04:09.859 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2023-06-22T17:04:09.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-22T17:04:09.989 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.987+0000 7fddcab25700 1 Processor -- start 2023-06-22T17:04:09.989 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.987+0000 7fddcab25700 1 -- start start 2023-06-22T17:04:09.990 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.988+0000 7fddcab25700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fddc4142de0 0x7fddc40b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.990 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.988+0000 7fddcab25700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 0x7fddc40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.990 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.988+0000 7fddcab25700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc40b2780 con 0x7fddc40b52f0 2023-06-22T17:04:09.990 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.988+0000 7fddcab25700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc40cec20 con 0x7fddc4142de0 2023-06-22T17:04:09.990 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.988+0000 7fddcab25700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40b5ae0 con 0x7fddc40b2310 2023-06-22T17:04:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.988+0000 7fddca324700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fddc40b52f0 0x7fddc415d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57028/0 (socket says 172.21.15.47:57028) 2023-06-22T17:04:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.988+0000 7fddca324700 1 -- 172.21.15.47:0/2311527625 learned_addr learned my addr 172.21.15.47:0/2311527625 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.988+0000 7fddc9b23700 1 --2- 172.21.15.47:0/2311527625 >> v2:172.21.15.202:3300/0 conn(0x7fddc4142de0 0x7fddc40b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.988+0000 7fddc9322700 1 --2- 172.21.15.47:0/2311527625 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 0x7fddc40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.988+0000 7fddc8b21700 1 -- 172.21.15.47:0/2311527625 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194399544 0 0) 0x7fddc40b2780 con 0x7fddc40b52f0 2023-06-22T17:04:09.991 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.989+0000 7fddc8b21700 1 -- 172.21.15.47:0/2311527625 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddac003570 con 0x7fddc40b52f0 2023-06-22T17:04:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.989+0000 7fddc9b23700 1 -- 172.21.15.47:0/2311527625 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 msgr2=0x7fddc40b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.989+0000 7fddc9b23700 1 --2- 172.21.15.47:0/2311527625 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 0x7fddc40b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.989+0000 7fddc9b23700 1 -- 172.21.15.47:0/2311527625 >> v1:172.21.15.47:6789/0 conn(0x7fddc40b52f0 legacy=0x7fddc415d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.989+0000 7fddc9b23700 1 -- 172.21.15.47:0/2311527625 --> v2:172.21.15.202:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40b5dc0 con 0x7fddc4142de0 2023-06-22T17:04:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.989+0000 7fddc9b23700 1 --2- 172.21.15.47:0/2311527625 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fddc4142de0 0x7fddc40b1dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0098e0 tx=0x7fddbc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ae028a719af6f1 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:09.993 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.989+0000 7fddc8b21700 1 -- 172.21.15.47:0/2311527625 <== mon.1 v2:172.21.15.202:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc011630 con 0x7fddc4142de0 2023-06-22T17:04:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.989+0000 7fddc8b21700 1 -- 172.21.15.47:0/2311527625 <== mon.1 v2:172.21.15.202:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc011790 con 0x7fddc4142de0 2023-06-22T17:04:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.989+0000 7fddc8b21700 1 -- 172.21.15.47:0/2311527625 <== mon.1 v2:172.21.15.202:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc011b30 con 0x7fddc4142de0 2023-06-22T17:04:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.990+0000 7fddcab25700 1 -- 172.21.15.47:0/2311527625 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fddc4142de0 msgr2=0x7fddc40b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.990+0000 7fddcab25700 1 --2- 172.21.15.47:0/2311527625 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fddc4142de0 0x7fddc40b1dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0098e0 tx=0x7fddbc0091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.990+0000 7fddcab25700 1 -- 172.21.15.47:0/2311527625 shutdown_connections 2023-06-22T17:04:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.990+0000 7fddcab25700 1 --2- 172.21.15.47:0/2311527625 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fddc4142de0 0x7fddc40b1dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.990+0000 7fddcab25700 1 --2- 172.21.15.47:0/2311527625 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 0x7fddc40b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.994 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.990+0000 7fddcab25700 1 -- 172.21.15.47:0/2311527625 >> 172.21.15.47:0/2311527625 conn(0x7fddc4070290 msgr2=0x7fddc4070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.990+0000 7fddcab25700 1 -- 172.21.15.47:0/2311527625 shutdown_connections 2023-06-22T17:04:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.990+0000 7fddcab25700 1 -- 172.21.15.47:0/2311527625 wait complete. 2023-06-22T17:04:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.991+0000 7fddcab25700 1 Processor -- start 2023-06-22T17:04:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.991+0000 7fddcab25700 1 -- start start 2023-06-22T17:04:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddcab25700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 0x7fddc4140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddcab25700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddc40b52f0 0x7fddc415e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddcab25700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fddc4142de0 0x7fddc415ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddcab25700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc415d6f0 con 0x7fddc40b52f0 2023-06-22T17:04:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddcab25700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40c6f30 con 0x7fddc4142de0 2023-06-22T17:04:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddcab25700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc415d8d0 con 0x7fddc40b2310 2023-06-22T17:04:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9322700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddc40b52f0 0x7fddc415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9322700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddc40b52f0 0x7fddc415e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51316/0 (socket says 172.21.15.47:51316) 2023-06-22T17:04:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9322700 1 -- 172.21.15.47:0/2108086137 learned_addr learned my addr 172.21.15.47:0/2108086137 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9b23700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 0x7fddc4140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.998 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddca324700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fddc4142de0 0x7fddc415ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9322700 1 -- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 msgr2=0x7fddc4140b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9322700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 0x7fddc4140b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9322700 1 -- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fddc4142de0 msgr2=0x7fddc415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9322700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fddc4142de0 0x7fddc415ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9322700 1 -- 172.21.15.47:0/2108086137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc009420 con 0x7fddc40b52f0 2023-06-22T17:04:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9b23700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 0x7fddc4140b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T17:04:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddca324700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fddc4142de0 0x7fddc415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T17:04:09.999 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc9322700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddc40b52f0 0x7fddc415e690 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fddb800ec80 tx=0x7fddb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc1ffb700 1 -- 172.21.15.47:0/2108086137 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800c970 con 0x7fddc40b52f0 2023-06-22T17:04:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc1ffb700 1 -- 172.21.15.47:0/2108086137 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb800cad0 con 0x7fddc40b52f0 2023-06-22T17:04:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddcab25700 1 -- 172.21.15.47:0/2108086137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc415dbb0 con 0x7fddc40b52f0 2023-06-22T17:04:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.992+0000 7fddc1ffb700 1 -- 172.21.15.47:0/2108086137 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb800ce70 con 0x7fddc40b52f0 2023-06-22T17:04:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.993+0000 7fddcab25700 1 -- 172.21.15.47:0/2108086137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc4141060 con 0x7fddc40b52f0 2023-06-22T17:04:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.994+0000 7fddcab25700 1 -- 172.21.15.47:0/2108086137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc415dda0 con 0x7fddc40b52f0 2023-06-22T17:04:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.994+0000 7fddc1ffb700 1 -- 172.21.15.47:0/2108086137 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fddb8013680 con 0x7fddc40b52f0 2023-06-22T17:04:10.000 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.994+0000 7fddc1ffb700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fddb0066e90 0x7fddb0069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.001 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.995+0000 7fddc1ffb700 1 -- 172.21.15.47:0/2108086137 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fddb8084530 con 0x7fddc40b52f0 2023-06-22T17:04:10.001 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.995+0000 7fddc9b23700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fddb0066e90 0x7fddb0069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.001 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:09.995+0000 7fddc9b23700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fddb0066e90 0x7fddb0069340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0091e0 tx=0x7fddbc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:10.001 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.000+0000 7fddc1ffb700 1 -- 172.21.15.47:0/2108086137 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fddb8051a50 con 0x7fddc40b52f0 2023-06-22T17:04:10.195 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.193+0000 7fddcab25700 1 -- 172.21.15.47:0/2108086137 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddc4088a10 con 0x7fddc40b52f0 2023-06-22T17:04:10.196 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.194+0000 7fddc1ffb700 1 -- 172.21.15.47:0/2108086137 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13647 (secure 0 0 0) 0x7fddb8055070 con 0x7fddc40b52f0 2023-06-22T17:04:10.196 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:04:10.196 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"fsid":"61e931aa-8dfe-4361-8a29-1d30432b403b","created":"2023-06-22T17:03:48.391484+0000","modified":"2023-06-22T17:04:09.415858+0000","last_up_change":"2023-06-22T17:03:59.369744+0000","last_in_change":"2023-06-22T17:03:50.761615+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-22T17:04:00.277500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-22T17:04:06.740390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"191ca9cf-dc97-4325-b63b-a3e6a66b71b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":87966},{"type":"v1","addr":"172.21.15.47:6809","nonce":87966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":87966},{"type":"v1","addr":"172.21.15.47:6811","nonce":87966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":87966},{"type":"v1","addr":"172.21.15.47:6815","nonce":87966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":87966},{"type":"v1","addr":"172.21.15.47:6813","nonce":87966}]},"public_addr":"172.21.15.47:6809/87966","cluster_addr":"172.21.15.47:6811/87966","heartbeat_back_addr":"172.21.15.47:6815/87966","heartbeat_front_addr":"172.21.15.47:6813/87966","state":["exists","up"]},{"osd":1,"uuid":"d426e8cd-b1e6-40a3-9e65-cf52fac95a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":87978},{"type":"v1","addr":"172.21.15.47:6825","nonce":87978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":87978},{"type":"v1","addr":"172.21.15.47:6827","nonce":87978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":87978},{"type":"v1","addr":"172.21.15.47:6831","nonce":87978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":87978},{"type":"v1","addr":"172.21.15.47:6829","nonce":87978}]},"public_addr":"172.21.15.47:6825/87978","cluster_addr":"172.21.15.47:6827/87978","heartbeat_back_addr":"172.21.15.47:6831/87978","heartbeat_front_addr":"172.21.15.47:6829/87978","state":["exists","up"]},{"osd":2,"uuid":"487a8fca-1da6-4cb3-a73c-abc6e9e2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":87967},{"type":"v1","addr":"172.21.15.47:6801","nonce":87967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":87967},{"type":"v1","addr":"172.21.15.47:6803","nonce":87967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":87967},{"type":"v1","addr":"172.21.15.47:6807","nonce":87967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":87967},{"type":"v1","addr":"172.21.15.47:6805","nonce":87967}]},"public_addr":"172.21.15.47:6801/87967","cluster_addr":"172.21.15.47:6803/87967","heartbeat_back_addr":"172.21.15.47:6807/87967","heartbeat_front_addr":"172.21.15.47:6805/87967","state":["exists","up"]},{"osd":3,"uuid":"7089499f-dc69-4d6b-a6c3-5b30404ab1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":87968},{"type":"v1","addr":"172.21.15.47:6817","nonce":87968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":87968},{"type":"v1","addr":"172.21.15.47:6819","nonce":87968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":87968},{"type":"v1","addr":"172.21.15.47:6823","nonce":87968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":87968},{"type":"v1","addr":"172.21.15.47:6821","nonce":87968}]},"public_addr":"172.21.15.47:6817/87968","cluster_addr":"172.21.15.47:6819/87968","heartbeat_back_addr":"172.21.15.47:6823/87968","heartbeat_front_addr":"172.21.15.47:6821/87968","state":["exists","up"]},{"osd":4,"uuid":"f8b34dee-7945-42fd-9e58-98e5fe196b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6800","nonce":88201},{"type":"v1","addr":"172.21.15.202:6801","nonce":88201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6802","nonce":88201},{"type":"v1","addr":"172.21.15.202:6803","nonce":88201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6806","nonce":88201},{"type":"v1","addr":"172.21.15.202:6807","nonce":88201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6804","nonce":88201},{"type":"v1","addr":"172.21.15.202:6805","nonce":88201}]},"public_addr":"172.21.15.202:6801/88201","cluster_addr":"172.21.15.202:6803/88201","heartbeat_back_addr":"172.21.15.202:6807/88201","heartbeat_front_addr":"172.21.15.202:6805/88201","state":["exists","up"]},{"osd":5,"uuid":"9583b379-49e1-4b6c-adf5-6f19322cc76c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6808","nonce":88199},{"type":"v1","addr":"172.21.15.202:6809","nonce":88199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6810","nonce":88199},{"type":"v1","addr":"172.21.15.202:6812","nonce":88199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6818","nonce":88199},{"type":"v1","addr":"172.21.15.202:6820","nonce":88199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6814","nonce":88199},{"type":"v1","addr":"172.21.15.202:6816","nonce":88199}]},"public_addr":"172.21.15.202:6809/88199","cluster_addr":"172.21.15.202:6812/88199","heartbeat_back_addr":"172.21.15.202:6820/88199","heartbeat_front_addr":"172.21.15.202:6816/88199","state":["exists","up"]},{"osd":6,"uuid":"d11633b5-5a36-441b-93c7-7c0b434f0dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6811","nonce":88200},{"type":"v1","addr":"172.21.15.202:6813","nonce":88200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6815","nonce":88200},{"type":"v1","addr":"172.21.15.202:6817","nonce":88200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6822","nonce":88200},{"type":"v1","addr":"172.21.15.202:6823","nonce":88200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6819","nonce":88200},{"type":"v1","addr":"172.21.15.202:6821","nonce":88200}]},"public_addr":"172.21.15.202:6813/88200","cluster_addr":"172.21.15.202:6817/88200","heartbeat_back_addr":"172.21.15.202:6823/88200","heartbeat_front_addr":"172.21.15.202:6821/88200","state":["exists","up"]},{"osd":7,"uuid":"edd9bf42-9566-4d07-902b-08cf16cb42a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6824","nonce":88230},{"type":"v1","addr":"172.21.15.202:6825","nonce":88230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6826","nonce":88230},{"type":"v1","addr":"172.21.15.202:6827","nonce":88230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6830","nonce":88230},{"type":"v1","addr":"172.21.15.202:6831","nonce":88230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6828","nonce":88230},{"type":"v1","addr":"172.21.15.202:6829","nonce":88230}]},"public_addr":"172.21.15.202:6825/88230","cluster_addr":"172.21.15.202:6827/88230","heartbeat_back_addr":"172.21.15.202:6831/88230","heartbeat_front_addr":"172.21.15.202:6829/88230","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.975830+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.942848+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.958568+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.972986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.946508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.938793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.960236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.931356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T17:04:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 -- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fddb0066e90 msgr2=0x7fddb0069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fddb0066e90 0x7fddb0069340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0091e0 tx=0x7fddbc017000 comp rx=0 tx=0).stop 2023-06-22T17:04:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 -- 172.21.15.47:0/2108086137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddc40b52f0 msgr2=0x7fddc415e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddc40b52f0 0x7fddc415e690 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fddb800ec80 tx=0x7fddb80091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 -- 172.21.15.47:0/2108086137 shutdown_connections 2023-06-22T17:04:10.199 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fddb0066e90 0x7fddb0069340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.199 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fddc4142de0 0x7fddc415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.199 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fddc40b2310 0x7fddc4140b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.199 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 --2- 172.21.15.47:0/2108086137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddc40b52f0 0x7fddc415e690 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.199 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 -- 172.21.15.47:0/2108086137 >> 172.21.15.47:0/2108086137 conn(0x7fddc4070290 msgr2=0x7fddc4141e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:10.199 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 -- 172.21.15.47:0/2108086137 shutdown_connections 2023-06-22T17:04:10.199 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.196+0000 7fddcab25700 1 -- 172.21.15.47:0/2108086137 wait complete. 2023-06-22T17:04:10.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-22T17:04:00.277500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-22T17:04:06.740390+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-06-22T17:04:10.210 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-22T17:04:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.330+0000 7f7d708b4700 1 Processor -- start 2023-06-22T17:04:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.330+0000 7f7d708b4700 1 -- start start 2023-06-22T17:04:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7d6c0b23d0 0x7f7d6c1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d6c0b00e0 0x7f7d6c0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6c0b0a80 con 0x7f7d6c159310 2023-06-22T17:04:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d6c0b2e10 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c0b15c0 con 0x7f7d6c0b23d0 2023-06-22T17:04:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d6b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d6c0b00e0 0x7f7d6c0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d6b59e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d6c0b00e0 0x7f7d6c0b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51326/0 (socket says 172.21.15.47:51326) 2023-06-22T17:04:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d6b59e700 1 -- 172.21.15.47:0/1150045271 learned_addr learned my addr 172.21.15.47:0/1150045271 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d6a59c700 1 --2- 172.21.15.47:0/1150045271 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7d6c0b23d0 0x7f7d6c1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d6b59e700 1 -- 172.21.15.47:0/1150045271 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7d6c0b23d0 msgr2=0x7f7d6c1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d6b59e700 1 --2- 172.21.15.47:0/1150045271 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7d6c0b23d0 0x7f7d6c1596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d6b59e700 1 -- 172.21.15.47:0/1150045271 >> v1:172.21.15.202:6789/0 conn(0x7f7d6c159310 legacy=0x7f7d6c15dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d6b59e700 1 -- 172.21.15.47:0/1150045271 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c0cb7d0 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d6b59e700 1 --2- 172.21.15.47:0/1150045271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 0x7f7d6c0b0540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d640099b0 tx=0x7f7d64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd291b8d36ff3538 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d69d9b700 1 -- 172.21.15.47:0/1150045271 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400b860 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d69d9b700 1 -- 172.21.15.47:0/1150045271 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6400b9c0 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d69d9b700 1 -- 172.21.15.47:0/1150045271 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400bd60 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 -- 172.21.15.47:0/1150045271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 msgr2=0x7f7d6c0b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 --2- 172.21.15.47:0/1150045271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 0x7f7d6c0b0540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d640099b0 tx=0x7f7d64009250 comp rx=0 tx=0).stop 2023-06-22T17:04:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 -- 172.21.15.47:0/1150045271 shutdown_connections 2023-06-22T17:04:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 --2- 172.21.15.47:0/1150045271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 0x7f7d6c0b0540 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 --2- 172.21.15.47:0/1150045271 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7d6c0b23d0 0x7f7d6c1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 -- 172.21.15.47:0/1150045271 >> 172.21.15.47:0/1150045271 conn(0x7f7d6c070290 msgr2=0x7f7d6c070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 -- 172.21.15.47:0/1150045271 shutdown_connections 2023-06-22T17:04:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.332+0000 7f7d708b4700 1 -- 172.21.15.47:0/1150045271 wait complete. 2023-06-22T17:04:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.333+0000 7f7d708b4700 1 Processor -- start 2023-06-22T17:04:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d708b4700 1 -- start start 2023-06-22T17:04:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d708b4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 0x7f7d6c147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d708b4700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7d6c159310 0x7f7d6c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d708b4700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7d6c15d250 0x7f7d6c15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d708b4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c15dce0 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d708b4700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c15d720 con 0x7f7d6c159310 2023-06-22T17:04:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d708b4700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c15da00 con 0x7f7d6c15d250 2023-06-22T17:04:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d6ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 0x7f7d6c147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d6ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 0x7f7d6c147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51342/0 (socket says 172.21.15.47:51342) 2023-06-22T17:04:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d6ad9d700 1 -- 172.21.15.47:0/2622444236 learned_addr learned my addr 172.21.15.47:0/2622444236 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.334+0000 7f7d6a59c700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7d6c159310 0x7f7d6c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d6b59e700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7d6c15d250 0x7f7d6c15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d6ad9d700 1 -- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7d6c15d250 msgr2=0x7f7d6c15ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d6ad9d700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7d6c15d250 0x7f7d6c15ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d6ad9d700 1 -- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7d6c159310 msgr2=0x7f7d6c15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d6ad9d700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7d6c159310 0x7f7d6c15cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d6ad9d700 1 -- 172.21.15.47:0/2622444236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d640094f0 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d6ad9d700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 0x7f7d6c147750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c8e0 tx=0x7f7d5c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d62ffd700 1 -- 172.21.15.47:0/2622444236 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c00ce80 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d62ffd700 1 -- 172.21.15.47:0/2622444236 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c00a7e0 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d708b4700 1 -- 172.21.15.47:0/2622444236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c0c40c0 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d62ffd700 1 -- 172.21.15.47:0/2622444236 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c00aba0 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.335+0000 7f7d708b4700 1 -- 172.21.15.47:0/2622444236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c157060 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.336+0000 7f7d708b4700 1 -- 172.21.15.47:0/2622444236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6c0c3f20 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.336+0000 7f7d62ffd700 1 -- 172.21.15.47:0/2622444236 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f7d5c003030 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.337+0000 7f7d62ffd700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f7d4c066e90 0x7f7d4c069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.337+0000 7f7d62ffd700 1 -- 172.21.15.47:0/2622444236 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7d5c084100 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.337+0000 7f7d6a59c700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f7d4c066e90 0x7f7d4c069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.337+0000 7f7d6a59c700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f7d4c066e90 0x7f7d4c069340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0afff0 tx=0x7f7d54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.341+0000 7f7d62ffd700 1 -- 172.21.15.47:0/2622444236 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f7d6c0c3f20 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.540+0000 7f7d708b4700 1 -- 172.21.15.47:0/2622444236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d6c0b0a80 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.541+0000 7f7d62ffd700 1 -- 172.21.15.47:0/2622444236 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d6c0b0a80 con 0x7f7d6c0b00e0 2023-06-22T17:04:10.543 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2023-06-22T17:04:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 -- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f7d4c066e90 msgr2=0x7f7d4c069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f7d4c066e90 0x7f7d4c069340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0afff0 tx=0x7f7d54008040 comp rx=0 tx=0).stop 2023-06-22T17:04:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 -- 172.21.15.47:0/2622444236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 msgr2=0x7f7d6c147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 0x7f7d6c147750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c8e0 tx=0x7f7d5c001710 comp rx=0 tx=0).stop 2023-06-22T17:04:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 -- 172.21.15.47:0/2622444236 shutdown_connections 2023-06-22T17:04:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f7d4c066e90 0x7f7d4c069340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f7d6c15d250 0x7f7d6c15ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d6c0b00e0 0x7f7d6c147750 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 --2- 172.21.15.47:0/2622444236 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f7d6c159310 0x7f7d6c15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 -- 172.21.15.47:0/2622444236 >> 172.21.15.47:0/2622444236 conn(0x7f7d6c070290 msgr2=0x7f7d6c0b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 -- 172.21.15.47:0/2622444236 shutdown_connections 2023-06-22T17:04:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.543+0000 7f7d708b4700 1 -- 172.21.15.47:0/2622444236 wait complete. 2023-06-22T17:04:10.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-22T17:04:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.677+0000 7f10898b0700 1 Processor -- start 2023-06-22T17:04:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.677+0000 7f10898b0700 1 -- start start 2023-06-22T17:04:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f10898b0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1084156ff0 0x7f108415a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f10898b0700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f108415d160 0x7f10840d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f10898b0700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10840cb1b0 con 0x7f10840d0870 2023-06-22T17:04:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f10898b0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108415ab30 con 0x7f1084156ff0 2023-06-22T17:04:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f10898b0700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108415d6a0 con 0x7f108415d160 2023-06-22T17:04:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f1082ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1084156ff0 0x7f108415a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f1082ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1084156ff0 0x7f108415a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51348/0 (socket says 172.21.15.47:51348) 2023-06-22T17:04:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f1082ffd700 1 -- 172.21.15.47:0/2411874342 learned_addr learned my addr 172.21.15.47:0/2411874342 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f10827fc700 1 --2- 172.21.15.47:0/2411874342 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f108415d160 0x7f10840d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f1082ffd700 1 -- 172.21.15.47:0/2411874342 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f108415d160 msgr2=0x7f10840d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f1082ffd700 1 --2- 172.21.15.47:0/2411874342 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f108415d160 0x7f10840d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f1082ffd700 1 -- 172.21.15.47:0/2411874342 >> v1:172.21.15.202:6789/0 conn(0x7f10840d0870 legacy=0x7f10840ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f1082ffd700 1 -- 172.21.15.47:0/2411874342 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108415d980 con 0x7f1084156ff0 2023-06-22T17:04:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.678+0000 7f1082ffd700 1 --2- 172.21.15.47:0/2411874342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1084156ff0 0x7f108415a5f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10740099b0 tx=0x7f1074009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad449caa324091b server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f1081ffb700 1 -- 172.21.15.47:0/2411874342 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107400b860 con 0x7f1084156ff0 2023-06-22T17:04:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f1081ffb700 1 -- 172.21.15.47:0/2411874342 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107400b9c0 con 0x7f1084156ff0 2023-06-22T17:04:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f1081ffb700 1 -- 172.21.15.47:0/2411874342 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107400bd60 con 0x7f1084156ff0 2023-06-22T17:04:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f10898b0700 1 -- 172.21.15.47:0/2411874342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1084156ff0 msgr2=0x7f108415a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f10898b0700 1 --2- 172.21.15.47:0/2411874342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1084156ff0 0x7f108415a5f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10740099b0 tx=0x7f1074009250 comp rx=0 tx=0).stop 2023-06-22T17:04:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f10898b0700 1 -- 172.21.15.47:0/2411874342 shutdown_connections 2023-06-22T17:04:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f10898b0700 1 --2- 172.21.15.47:0/2411874342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1084156ff0 0x7f108415a5f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f10898b0700 1 --2- 172.21.15.47:0/2411874342 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f108415d160 0x7f10840d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f10898b0700 1 -- 172.21.15.47:0/2411874342 >> 172.21.15.47:0/2411874342 conn(0x7f1084070290 msgr2=0x7f1084070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f10898b0700 1 -- 172.21.15.47:0/2411874342 shutdown_connections 2023-06-22T17:04:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.679+0000 7f10898b0700 1 -- 172.21.15.47:0/2411874342 wait complete. 2023-06-22T17:04:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.680+0000 7f10898b0700 1 Processor -- start 2023-06-22T17:04:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10898b0700 1 -- start start 2023-06-22T17:04:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10898b0700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f10840d0870 0x7f10841540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10898b0700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1084156ff0 0x7f10840c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10898b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108415d160 0x7f10840b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10898b0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840b1080 con 0x7f108415d160 2023-06-22T17:04:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10898b0700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1084154610 con 0x7f1084156ff0 2023-06-22T17:04:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10898b0700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10840b0470 con 0x7f10840d0870 2023-06-22T17:04:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10837fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108415d160 0x7f10840b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10837fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108415d160 0x7f10840b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51356/0 (socket says 172.21.15.47:51356) 2023-06-22T17:04:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10837fe700 1 -- 172.21.15.47:0/3800258180 learned_addr learned my addr 172.21.15.47:0/3800258180 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f1082ffd700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f10840d0870 0x7f10841540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.681+0000 7f10827fc700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1084156ff0 0x7f10840c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f10837fe700 1 -- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f10840d0870 msgr2=0x7f10841540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f10837fe700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f10840d0870 0x7f10841540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f10837fe700 1 -- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1084156ff0 msgr2=0x7f10840c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f10837fe700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1084156ff0 0x7f10840c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f10837fe700 1 -- 172.21.15.47:0/3800258180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10740094f0 con 0x7f108415d160 2023-06-22T17:04:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f10837fe700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108415d160 0x7f10840b4260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c910 tx=0x7f107c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f107affd700 1 -- 172.21.15.47:0/3800258180 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c012630 con 0x7f108415d160 2023-06-22T17:04:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f107affd700 1 -- 172.21.15.47:0/3800258180 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c012790 con 0x7f108415d160 2023-06-22T17:04:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f10898b0700 1 -- 172.21.15.47:0/3800258180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10840b0750 con 0x7f108415d160 2023-06-22T17:04:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f107affd700 1 -- 172.21.15.47:0/3800258180 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c012b50 con 0x7f108415d160 2023-06-22T17:04:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.682+0000 7f10898b0700 1 -- 172.21.15.47:0/3800258180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10840b0d10 con 0x7f108415d160 2023-06-22T17:04:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.683+0000 7f10898b0700 1 -- 172.21.15.47:0/3800258180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10840b1230 con 0x7f108415d160 2023-06-22T17:04:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.684+0000 7f107affd700 1 -- 172.21.15.47:0/3800258180 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f107c01a3a0 con 0x7f108415d160 2023-06-22T17:04:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.684+0000 7f107affd700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f1064066e90 0x7f1064069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.684+0000 7f107affd700 1 -- 172.21.15.47:0/3800258180 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f107c0844c0 con 0x7f108415d160 2023-06-22T17:04:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.684+0000 7f1082ffd700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f1064066e90 0x7f1064069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.685+0000 7f1082ffd700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f1064066e90 0x7f1064069340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f10740099b0 tx=0x7f1074017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.689+0000 7f107affd700 1 -- 172.21.15.47:0/3800258180 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f10840b1230 con 0x7f108415d160 2023-06-22T17:04:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.881+0000 7f10898b0700 1 -- 172.21.15.47:0/3800258180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f10840b1230 con 0x7f108415d160 2023-06-22T17:04:10.882 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.881+0000 7f107affd700 1 -- 172.21.15.47:0/3800258180 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f10840b1230 con 0x7f108415d160 2023-06-22T17:04:10.883 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2023-06-22T17:04:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 -- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f1064066e90 msgr2=0x7f1064069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f1064066e90 0x7f1064069340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f10740099b0 tx=0x7f1074017000 comp rx=0 tx=0).stop 2023-06-22T17:04:10.885 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 -- 172.21.15.47:0/3800258180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108415d160 msgr2=0x7f10840b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:10.886 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108415d160 0x7f10840b4260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c910 tx=0x7f107c001710 comp rx=0 tx=0).stop 2023-06-22T17:04:10.886 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 -- 172.21.15.47:0/3800258180 shutdown_connections 2023-06-22T17:04:10.886 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f1064066e90 0x7f1064069340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.886 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f108415d160 0x7f10840b4260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.886 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f10840d0870 0x7f10841540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.887 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 --2- 172.21.15.47:0/3800258180 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f1084156ff0 0x7f10840c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:10.887 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 -- 172.21.15.47:0/3800258180 >> 172.21.15.47:0/3800258180 conn(0x7f1084070290 msgr2=0x7f10840c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:10.887 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 -- 172.21.15.47:0/3800258180 shutdown_connections 2023-06-22T17:04:10.887 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:10.883+0000 7f10898b0700 1 -- 172.21.15.47:0/3800258180 wait complete. 2023-06-22T17:04:10.896 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-22T17:04:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.014+0000 7f72d2a34700 1 Processor -- start 2023-06-22T17:04:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.014+0000 7f72d2a34700 1 -- start start 2023-06-22T17:04:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.015+0000 7f72d2a34700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f72cc0ad400 0x7f72cc06ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.015+0000 7f72d2a34700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 0x7f72cc0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.015+0000 7f72d2a34700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc1566a0 con 0x7f72cc1562d0 2023-06-22T17:04:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.015+0000 7f72d2a34700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc0b0c90 con 0x7f72cc0ad400 2023-06-22T17:04:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.015+0000 7f72d2a34700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0cce30 con 0x7f72cc0cc9b0 2023-06-22T17:04:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.015+0000 7f72d1231700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f72cc0ad400 0x7f72cc06ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.015+0000 7f72d2233700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 0x7f72cc0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.015+0000 7f72d1a32700 1 --1- >> v1:172.21.15.202:6789/0 conn(0x7f72cc1562d0 0x7f72cc0accf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.202:6789/0 says I am v1:172.21.15.47:51428/0 (socket says 172.21.15.47:51428) 2023-06-22T17:04:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.015+0000 7f72d1a32700 1 -- 172.21.15.47:0/2626783507 learned_addr learned my addr 172.21.15.47:0/2626783507 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d1231700 1 -- 172.21.15.47:0/2626783507 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 msgr2=0x7f72cc0c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d1231700 1 --2- 172.21.15.47:0/2626783507 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 0x7f72cc0c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d1231700 1 -- 172.21.15.47:0/2626783507 >> v1:172.21.15.202:6789/0 conn(0x7f72cc1562d0 legacy=0x7f72cc0accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d1231700 1 -- 172.21.15.47:0/2626783507 --> v2:172.21.15.202:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0b0080 con 0x7f72cc0ad400 2023-06-22T17:04:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d0a30700 1 -- 172.21.15.47:0/2626783507 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838412670 0 0) 0x7f72cc1566a0 con 0x7f72cc1562d0 2023-06-22T17:04:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d1231700 1 --2- 172.21.15.47:0/2626783507 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f72cc0ad400 0x7f72cc06ac30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f72c00098e0 tx=0x7f72c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1808e2a77e8d9539 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d0a30700 1 -- 172.21.15.47:0/2626783507 <== mon.1 v2:172.21.15.202:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c000f6b0 con 0x7f72cc0ad400 2023-06-22T17:04:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d0a30700 1 -- 172.21.15.47:0/2626783507 <== mon.1 v2:172.21.15.202:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c000f810 con 0x7f72cc0ad400 2023-06-22T17:04:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d0a30700 1 -- 172.21.15.47:0/2626783507 <== mon.1 v2:172.21.15.202:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c000fbb0 con 0x7f72cc0ad400 2023-06-22T17:04:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d2a34700 1 -- 172.21.15.47:0/2626783507 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f72cc0ad400 msgr2=0x7f72cc06ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.016+0000 7f72d2a34700 1 --2- 172.21.15.47:0/2626783507 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f72cc0ad400 0x7f72cc06ac30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f72c00098e0 tx=0x7f72c00091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.017+0000 7f72d2a34700 1 -- 172.21.15.47:0/2626783507 shutdown_connections 2023-06-22T17:04:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.017+0000 7f72d2a34700 1 --2- 172.21.15.47:0/2626783507 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 0x7f72cc0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.017+0000 7f72d2a34700 1 --2- 172.21.15.47:0/2626783507 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f72cc0ad400 0x7f72cc06ac30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.017+0000 7f72d2a34700 1 -- 172.21.15.47:0/2626783507 >> 172.21.15.47:0/2626783507 conn(0x7f72cc067410 msgr2=0x7f72cc067810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.017+0000 7f72d2a34700 1 -- 172.21.15.47:0/2626783507 shutdown_connections 2023-06-22T17:04:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.017+0000 7f72d2a34700 1 -- 172.21.15.47:0/2626783507 wait complete. 2023-06-22T17:04:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.018+0000 7f72d2a34700 1 Processor -- start 2023-06-22T17:04:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.018+0000 7f72d2a34700 1 -- start start 2023-06-22T17:04:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.018+0000 7f72d2a34700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f72cc0ad400 0x7f72cc154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.018+0000 7f72d2a34700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 0x7f72cc0a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.018+0000 7f72d2a34700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc1562d0 0x7f72cc0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.018+0000 7f72d2a34700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0a6330 con 0x7f72cc1562d0 2023-06-22T17:04:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.018+0000 7f72d2a34700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0a5420 con 0x7f72cc0ad400 2023-06-22T17:04:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.018+0000 7f72d2a34700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc150290 con 0x7f72cc0cc9b0 2023-06-22T17:04:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2233700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc1562d0 0x7f72cc0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2233700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc1562d0 0x7f72cc0caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51372/0 (socket says 172.21.15.47:51372) 2023-06-22T17:04:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2233700 1 -- 172.21.15.47:0/3360510859 learned_addr learned my addr 172.21.15.47:0/3360510859 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d1231700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 0x7f72cc0a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d1a32700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f72cc0ad400 0x7f72cc154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2233700 1 -- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 msgr2=0x7f72cc0a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2233700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 0x7f72cc0a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2233700 1 -- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f72cc0ad400 msgr2=0x7f72cc154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2233700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f72cc0ad400 0x7f72cc154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2233700 1 -- 172.21.15.47:0/3360510859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0009420 con 0x7f72cc1562d0 2023-06-22T17:04:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d1231700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 0x7f72cc0a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T17:04:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2233700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc1562d0 0x7f72cc0caaa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009e90 tx=0x7f72c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72c9ffb700 1 -- 172.21.15.47:0/3360510859 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400cc20 con 0x7f72cc1562d0 2023-06-22T17:04:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72c9ffb700 1 -- 172.21.15.47:0/3360510859 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c400f040 con 0x7f72cc1562d0 2023-06-22T17:04:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72c9ffb700 1 -- 172.21.15.47:0/3360510859 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4014480 con 0x7f72cc1562d0 2023-06-22T17:04:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2a34700 1 -- 172.21.15.47:0/3360510859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc153bc0 con 0x7f72cc1562d0 2023-06-22T17:04:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.019+0000 7f72d2a34700 1 -- 172.21.15.47:0/3360510859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc0a5950 con 0x7f72cc1562d0 2023-06-22T17:04:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.020+0000 7f72d2a34700 1 -- 172.21.15.47:0/3360510859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc0cb7b0 con 0x7f72cc1562d0 2023-06-22T17:04:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.021+0000 7f72c9ffb700 1 -- 172.21.15.47:0/3360510859 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f72c4014830 con 0x7f72cc1562d0 2023-06-22T17:04:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.021+0000 7f72c9ffb700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f72b8066e90 0x7f72b8069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.021+0000 7f72c9ffb700 1 -- 172.21.15.47:0/3360510859 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f72c4084c50 con 0x7f72cc1562d0 2023-06-22T17:04:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.022+0000 7f72d1a32700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f72b8066e90 0x7f72b8069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.022+0000 7f72d1a32700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f72b8066e90 0x7f72b8069340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00c8f0 tx=0x7f72bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.026+0000 7f72c9ffb700 1 -- 172.21.15.47:0/3360510859 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f72c408a020 con 0x7f72cc1562d0 2023-06-22T17:04:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.222+0000 7f72d2a34700 1 -- 172.21.15.47:0/3360510859 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f72cc06bb10 con 0x7f72cc1562d0 2023-06-22T17:04:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.223+0000 7f72c9ffb700 1 -- 172.21.15.47:0/3360510859 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7f72c4018090 con 0x7f72cc1562d0 2023-06-22T17:04:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:04:11.224 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2023-06-22T17:04:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2023-06-22T17:04:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2023-06-22T17:04:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2023-06-22T17:04:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2023-06-22T17:04:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2023-06-22T17:04:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 2, 2023-06-22T17:04:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": "rbd" 2023-06-22T17:04:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: } 2023-06-22T17:04:11.225 INFO:teuthology.orchestra.run.smithi047.stdout:] 2023-06-22T17:04:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 -- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f72b8066e90 msgr2=0x7f72b8069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f72b8066e90 0x7f72b8069340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00c8f0 tx=0x7f72bc009040 comp rx=0 tx=0).stop 2023-06-22T17:04:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 -- 172.21.15.47:0/3360510859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc1562d0 msgr2=0x7f72cc0caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc1562d0 0x7f72cc0caaa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009e90 tx=0x7f72c400b040 comp rx=0 tx=0).stop 2023-06-22T17:04:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 -- 172.21.15.47:0/3360510859 shutdown_connections 2023-06-22T17:04:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f72b8066e90 0x7f72b8069340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72cc1562d0 0x7f72cc0caaa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f72cc0ad400 0x7f72cc154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 --2- 172.21.15.47:0/3360510859 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f72cc0cc9b0 0x7f72cc0a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 -- 172.21.15.47:0/3360510859 >> 172.21.15.47:0/3360510859 conn(0x7f72cc067410 msgr2=0x7f72cc151b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 -- 172.21.15.47:0/3360510859 shutdown_connections 2023-06-22T17:04:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.225+0000 7f72d2a34700 1 -- 172.21.15.47:0/3360510859 wait complete. 2023-06-22T17:04:11.238 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-22T17:04:11.238 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-22T17:04:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.361+0000 7f88b714a700 1 Processor -- start 2023-06-22T17:04:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.361+0000 7f88b714a700 1 -- start start 2023-06-22T17:04:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b714a700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f88b015d340 0x7f88b00cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b714a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88b015d710 0x7f88b015c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b714a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b015ba70 con 0x7f88b00cb9b0 2023-06-22T17:04:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b714a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b00cdc00 con 0x7f88b015d710 2023-06-22T17:04:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b714a700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b015ae60 con 0x7f88b015d340 2023-06-22T17:04:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b6949700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88b015d710 0x7f88b015c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b5947700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f88b00cb9b0 0x7f88b015bf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57038/0 (socket says 172.21.15.47:57038) 2023-06-22T17:04:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b5947700 1 -- 172.21.15.47:0/4187027757 learned_addr learned my addr 172.21.15.47:0/4187027757 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b6148700 1 --2- 172.21.15.47:0/4187027757 >> v2:172.21.15.202:3300/0 conn(0x7f88b015d340 0x7f88b00cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b6949700 1 -- 172.21.15.47:0/4187027757 >> v2:172.21.15.202:3300/0 conn(0x7f88b015d340 msgr2=0x7f88b00cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b6949700 1 --2- 172.21.15.47:0/4187027757 >> v2:172.21.15.202:3300/0 conn(0x7f88b015d340 0x7f88b00cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b6949700 1 -- 172.21.15.47:0/4187027757 >> v1:172.21.15.47:6789/0 conn(0x7f88b00cb9b0 legacy=0x7f88b015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b5146700 1 -- 172.21.15.47:0/4187027757 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374829631 0 0) 0x7f88b015ba70 con 0x7f88b00cb9b0 2023-06-22T17:04:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.362+0000 7f88b6949700 1 -- 172.21.15.47:0/4187027757 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b015b140 con 0x7f88b015d710 2023-06-22T17:04:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b6949700 1 --2- 172.21.15.47:0/4187027757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 0x7f88b015c610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f88b00cdf60 tx=0x7f88a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda66b2cdaf6c7ea server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b5146700 1 -- 172.21.15.47:0/4187027757 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a800b6b0 con 0x7f88b015d710 2023-06-22T17:04:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b5146700 1 -- 172.21.15.47:0/4187027757 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a800b810 con 0x7f88b015d710 2023-06-22T17:04:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b5146700 1 -- 172.21.15.47:0/4187027757 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a800bbb0 con 0x7f88b015d710 2023-06-22T17:04:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b714a700 1 -- 172.21.15.47:0/4187027757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 msgr2=0x7f88b015c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b714a700 1 --2- 172.21.15.47:0/4187027757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 0x7f88b015c610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f88b00cdf60 tx=0x7f88a80091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b714a700 1 -- 172.21.15.47:0/4187027757 shutdown_connections 2023-06-22T17:04:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b714a700 1 --2- 172.21.15.47:0/4187027757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 0x7f88b015c610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b714a700 1 --2- 172.21.15.47:0/4187027757 >> v2:172.21.15.202:3300/0 conn(0x7f88b015d340 0x7f88b00cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b714a700 1 -- 172.21.15.47:0/4187027757 >> 172.21.15.47:0/4187027757 conn(0x7f88b0070290 msgr2=0x7f88b0070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b714a700 1 -- 172.21.15.47:0/4187027757 shutdown_connections 2023-06-22T17:04:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.363+0000 7f88b714a700 1 -- 172.21.15.47:0/4187027757 wait complete. 2023-06-22T17:04:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.364+0000 7f88b714a700 1 Processor -- start 2023-06-22T17:04:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.364+0000 7f88b714a700 1 -- start start 2023-06-22T17:04:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b714a700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f88b00cb9b0 0x7f88b016a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b714a700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f88b015d340 0x7f88b00c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.371 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b714a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 0x7f88b00cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.371 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b714a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00d0b60 con 0x7f88b015d710 2023-06-22T17:04:11.371 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b714a700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00ce780 con 0x7f88b00cb9b0 2023-06-22T17:04:11.371 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b714a700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b016adc0 con 0x7f88b015d340 2023-06-22T17:04:11.371 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b6949700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 0x7f88b00cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b6949700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 0x7f88b00cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51384/0 (socket says 172.21.15.47:51384) 2023-06-22T17:04:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b6949700 1 -- 172.21.15.47:0/2744155829 learned_addr learned my addr 172.21.15.47:0/2744155829 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b6148700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f88b00cb9b0 0x7f88b016a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.365+0000 7f88b5947700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f88b015d340 0x7f88b00c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88b6949700 1 -- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f88b015d340 msgr2=0x7f88b00c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88b6949700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f88b015d340 0x7f88b00c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88b6949700 1 -- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f88b00cb9b0 msgr2=0x7f88b016a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88b6949700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f88b00cb9b0 0x7f88b016a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88b6949700 1 -- 172.21.15.47:0/2744155829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a8009420 con 0x7f88b015d710 2023-06-22T17:04:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88b6949700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 0x7f88b00cfea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88a800b0e0 tx=0x7f88a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88ae7fc700 1 -- 172.21.15.47:0/2744155829 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a800bad0 con 0x7f88b015d710 2023-06-22T17:04:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88ae7fc700 1 -- 172.21.15.47:0/2744155829 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a8025420 con 0x7f88b015d710 2023-06-22T17:04:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88b714a700 1 -- 172.21.15.47:0/2744155829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b00ce9d0 con 0x7f88b015d710 2023-06-22T17:04:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88ae7fc700 1 -- 172.21.15.47:0/2744155829 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a80257e0 con 0x7f88b015d710 2023-06-22T17:04:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.366+0000 7f88b714a700 1 -- 172.21.15.47:0/2744155829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b00cef90 con 0x7f88b015d710 2023-06-22T17:04:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.367+0000 7f88b714a700 1 -- 172.21.15.47:0/2744155829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b0088a40 con 0x7f88b015d710 2023-06-22T17:04:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.367+0000 7f88ae7fc700 1 -- 172.21.15.47:0/2744155829 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f88a8023070 con 0x7f88b015d710 2023-06-22T17:04:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.368+0000 7f88ae7fc700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f889c066e90 0x7f889c069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.368+0000 7f88ae7fc700 1 -- 172.21.15.47:0/2744155829 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f88a8085910 con 0x7f88b015d710 2023-06-22T17:04:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.368+0000 7f88b6148700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f889c066e90 0x7f889c069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.368+0000 7f88b6148700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f889c066e90 0x7f889c069340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88a000f6c0 tx=0x7f88a0007840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:11.375 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.372+0000 7f88ae7fc700 1 -- 172.21.15.47:0/2744155829 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f88a8052e30 con 0x7f88b015d710 2023-06-22T17:04:11.568 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:11.567+0000 7f88b714a700 1 -- 172.21.15.47:0/2744155829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f88b015b420 con 0x7f88b015d710 2023-06-22T17:04:12.539 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.538+0000 7f88ae7fc700 1 -- 172.21.15.47:0/2744155829 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7f88a8056450 con 0x7f88b015d710 2023-06-22T17:04:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.543+0000 7f88b714a700 1 -- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f889c066e90 msgr2=0x7f889c069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.543+0000 7f88b714a700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f889c066e90 0x7f889c069340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88a000f6c0 tx=0x7f88a0007840 comp rx=0 tx=0).stop 2023-06-22T17:04:12.543 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.543+0000 7f88b714a700 1 -- 172.21.15.47:0/2744155829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 msgr2=0x7f88b00cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:12.544 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.543+0000 7f88b714a700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 0x7f88b00cfea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88a800b0e0 tx=0x7f88a8017000 comp rx=0 tx=0).stop 2023-06-22T17:04:12.544 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.543+0000 7f88b714a700 1 -- 172.21.15.47:0/2744155829 shutdown_connections 2023-06-22T17:04:12.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.543+0000 7f88b714a700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f889c066e90 0x7f889c069340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:12.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.543+0000 7f88b714a700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b015d710 0x7f88b00cfea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:12.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.543+0000 7f88b714a700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f88b00cb9b0 0x7f88b016a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:12.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.543+0000 7f88b714a700 1 --2- 172.21.15.47:0/2744155829 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f88b015d340 0x7f88b00c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:12.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.543+0000 7f88b714a700 1 -- 172.21.15.47:0/2744155829 >> 172.21.15.47:0/2744155829 conn(0x7f88b0070290 msgr2=0x7f88b00b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:12.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.544+0000 7f88b714a700 1 -- 172.21.15.47:0/2744155829 shutdown_connections 2023-06-22T17:04:12.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.544+0000 7f88b714a700 1 -- 172.21.15.47:0/2744155829 wait complete. 2023-06-22T17:04:12.547 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2023-06-22T17:04:12.562 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-22T17:04:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.694+0000 7fcf0c868700 1 Processor -- start 2023-06-22T17:04:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.694+0000 7fcf0c868700 1 -- start start 2023-06-22T17:04:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf0c868700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 0x7fcf080d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf0c868700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fcf080b2010 0x7fcf080b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf0c868700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf080cedd0 con 0x7fcf08153a80 2023-06-22T17:04:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf0c868700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf080d14c0 con 0x7fcf080b2010 2023-06-22T17:04:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf0c868700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf080b29d0 con 0x7fcf08153e50 2023-06-22T17:04:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf05d9b700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 0x7fcf080d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf06d9d700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fcf080b2010 0x7fcf080b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf06d9d700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7fcf080b2010 0x7fcf080b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3300/0 says I am v2:172.21.15.47:57082/0 (socket says 172.21.15.47:57082) 2023-06-22T17:04:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf06d9d700 1 -- 172.21.15.47:0/2163201501 learned_addr learned my addr 172.21.15.47:0/2163201501 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf05d9b700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 0x7fcf080d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.202:3301/0 says I am v2:172.21.15.47:48480/0 (socket says 172.21.15.47:48480) 2023-06-22T17:04:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf0659c700 1 --1- 172.21.15.47:0/2163201501 >> v1:172.21.15.202:6789/0 conn(0x7fcf08153a80 0x7fcf0815a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.202:6789/0 says I am v1:172.21.15.47:51438/0 (socket says 172.21.15.47:51438) 2023-06-22T17:04:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf05d9b700 1 -- 172.21.15.47:0/2163201501 >> v2:172.21.15.202:3300/0 conn(0x7fcf080b2010 msgr2=0x7fcf080b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf05d9b700 1 --2- 172.21.15.47:0/2163201501 >> v2:172.21.15.202:3300/0 conn(0x7fcf080b2010 0x7fcf080b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf05d9b700 1 -- 172.21.15.47:0/2163201501 >> v1:172.21.15.202:6789/0 conn(0x7fcf08153a80 legacy=0x7fcf0815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf05d9b700 1 -- 172.21.15.47:0/2163201501 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0815aed0 con 0x7fcf08153e50 2023-06-22T17:04:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.695+0000 7fcf06d9d700 1 --2- 172.21.15.47:0/2163201501 >> v2:172.21.15.202:3300/0 conn(0x7fcf080b2010 0x7fcf080b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T17:04:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf05d9b700 1 --2- 172.21.15.47:0/2163201501 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 0x7fcf080d0590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0098e0 tx=0x7fcefc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0df52f74805129a server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0559a700 1 -- 172.21.15.47:0/2163201501 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00b640 con 0x7fcf08153e50 2023-06-22T17:04:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0559a700 1 -- 172.21.15.47:0/2163201501 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc00b7a0 con 0x7fcf08153e50 2023-06-22T17:04:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0559a700 1 -- 172.21.15.47:0/2163201501 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00bb40 con 0x7fcf08153e50 2023-06-22T17:04:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0c868700 1 -- 172.21.15.47:0/2163201501 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 msgr2=0x7fcf080d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0c868700 1 --2- 172.21.15.47:0/2163201501 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 0x7fcf080d0590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0098e0 tx=0x7fcefc0091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0c868700 1 -- 172.21.15.47:0/2163201501 shutdown_connections 2023-06-22T17:04:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0c868700 1 --2- 172.21.15.47:0/2163201501 >> v2:172.21.15.202:3300/0 conn(0x7fcf080b2010 0x7fcf080b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0c868700 1 --2- 172.21.15.47:0/2163201501 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 0x7fcf080d0590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0c868700 1 -- 172.21.15.47:0/2163201501 >> 172.21.15.47:0/2163201501 conn(0x7fcf08070290 msgr2=0x7fcf08070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0c868700 1 -- 172.21.15.47:0/2163201501 shutdown_connections 2023-06-22T17:04:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.696+0000 7fcf0c868700 1 -- 172.21.15.47:0/2163201501 wait complete. 2023-06-22T17:04:12.703 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.697+0000 7fcf0c868700 1 Processor -- start 2023-06-22T17:04:12.703 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.697+0000 7fcf0c868700 1 -- start start 2023-06-22T17:04:12.703 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf0c868700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf080b2010 0x7fcf08158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:12.703 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf0c868700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fcf08153a80 0x7fcf08140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:12.703 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf0c868700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 0x7fcf081571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:12.703 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf0c868700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf08158740 con 0x7fcf080b2010 2023-06-22T17:04:12.704 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf0c868700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf08157b30 con 0x7fcf08153a80 2023-06-22T17:04:12.704 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf0c868700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf08157e10 con 0x7fcf08153e50 2023-06-22T17:04:12.704 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf0659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf080b2010 0x7fcf08158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:12.704 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf0659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf080b2010 0x7fcf08158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51390/0 (socket says 172.21.15.47:51390) 2023-06-22T17:04:12.704 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf0659c700 1 -- 172.21.15.47:0/3869346047 learned_addr learned my addr 172.21.15.47:0/3869346047 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:12.704 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf06d9d700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 0x7fcf081571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:12.704 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.698+0000 7fcf05d9b700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fcf08153a80 0x7fcf08140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcf0659c700 1 -- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 msgr2=0x7fcf081571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcf0659c700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 0x7fcf081571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcf0659c700 1 -- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fcf08153a80 msgr2=0x7fcf08140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcf0659c700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fcf08153a80 0x7fcf08140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcf0659c700 1 -- 172.21.15.47:0/3869346047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00009ae0 con 0x7fcf080b2010 2023-06-22T17:04:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcf0659c700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf080b2010 0x7fcf08158bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcef800c7c0 tx=0x7fcef8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcef67fc700 1 -- 172.21.15.47:0/3869346047 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8005a70 con 0x7fcf080b2010 2023-06-22T17:04:12.705 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcef67fc700 1 -- 172.21.15.47:0/3869346047 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef8005bd0 con 0x7fcf080b2010 2023-06-22T17:04:12.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcf0c868700 1 -- 172.21.15.47:0/3869346047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc009420 con 0x7fcf080b2010 2023-06-22T17:04:12.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcef67fc700 1 -- 172.21.15.47:0/3869346047 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80070d0 con 0x7fcf080b2010 2023-06-22T17:04:12.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.699+0000 7fcf0c868700 1 -- 172.21.15.47:0/3869346047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf081583d0 con 0x7fcf080b2010 2023-06-22T17:04:12.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.700+0000 7fcf0c868700 1 -- 172.21.15.47:0/3869346047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0804c1b0 con 0x7fcf080b2010 2023-06-22T17:04:12.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.700+0000 7fcef67fc700 1 -- 172.21.15.47:0/3869346047 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fcef8007480 con 0x7fcf080b2010 2023-06-22T17:04:12.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.701+0000 7fcef67fc700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcee8066ec0 0x7fcee8069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:12.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.701+0000 7fcef67fc700 1 -- 172.21.15.47:0/3869346047 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fcef8083840 con 0x7fcf080b2010 2023-06-22T17:04:12.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.701+0000 7fcf05d9b700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcee8066ec0 0x7fcee8069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:12.707 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.701+0000 7fcf05d9b700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcee8066ec0 0x7fcee8069370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00a010 tx=0x7fcefc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:12.707 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.705+0000 7fcef67fc700 1 -- 172.21.15.47:0/3869346047 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fcef8050bf0 con 0x7fcf080b2010 2023-06-22T17:04:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:12.901+0000 7fcf0c868700 1 -- 172.21.15.47:0/3869346047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcf080b6420 con 0x7fcf080b2010 2023-06-22T17:04:13.545 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.545+0000 7fcef67fc700 1 -- 172.21.15.47:0/3869346047 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v22) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcef80170d0 con 0x7fcf080b2010 2023-06-22T17:04:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 -- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcee8066ec0 msgr2=0x7fcee8069370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcee8066ec0 0x7fcee8069370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00a010 tx=0x7fcefc017000 comp rx=0 tx=0).stop 2023-06-22T17:04:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 -- 172.21.15.47:0/3869346047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf080b2010 msgr2=0x7fcf08158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf080b2010 0x7fcf08158bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcef800c7c0 tx=0x7fcef8007f70 comp rx=0 tx=0).stop 2023-06-22T17:04:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 -- 172.21.15.47:0/3869346047 shutdown_connections 2023-06-22T17:04:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7fcee8066ec0 0x7fcee8069370 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7fcf08153e50 0x7fcf081571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf080b2010 0x7fcf08158bd0 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcef800c7c0 tx=0x7fcef8007f70 comp rx=0 tx=0).stop 2023-06-22T17:04:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 --2- 172.21.15.47:0/3869346047 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7fcf08153a80 0x7fcf08140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 -- 172.21.15.47:0/3869346047 >> 172.21.15.47:0/3869346047 conn(0x7fcf08070290 msgr2=0x7fcf0815b760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.550+0000 7fcf0c868700 1 -- 172.21.15.47:0/3869346047 shutdown_connections 2023-06-22T17:04:13.553 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.551+0000 7fcf0c868700 1 -- 172.21.15.47:0/3869346047 wait complete. 2023-06-22T17:04:13.564 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2023-06-22T17:04:13.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-22T17:04:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.702+0000 7f1402624700 1 Processor -- start 2023-06-22T17:04:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.702+0000 7f1402624700 1 -- start start 2023-06-22T17:04:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f1402624700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f13fc153e50 0x7f13fc0d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f1402624700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc0b2010 0x7f13fc0b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f1402624700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13fc0cedd0 con 0x7f13fc153a80 2023-06-22T17:04:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f1402624700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13fc0d14c0 con 0x7f13fc153e50 2023-06-22T17:04:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f1402624700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0b29d0 con 0x7f13fc0b2010 2023-06-22T17:04:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13fbfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f13fc153a80 0x7f13fc15a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57054/0 (socket says 172.21.15.47:57054) 2023-06-22T17:04:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13fbfff700 1 -- 172.21.15.47:0/2867498422 learned_addr learned my addr 172.21.15.47:0/2867498422 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13fb7fe700 1 --2- 172.21.15.47:0/2867498422 >> v2:172.21.15.202:3300/0 conn(0x7f13fc153e50 0x7f13fc0d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f1400bc1700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc0b2010 0x7f13fc0b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756467780 0 0) 0x7f13fc0cedd0 con 0x7f13fc153a80 2023-06-22T17:04:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13e8003570 con 0x7f13fc153a80 2023-06-22T17:04:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1356665169 0 0) 0x7f13e8003570 con 0x7f13fc153a80 2023-06-22T17:04:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc0b2010 msgr2=0x7f13fc0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13faffd700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc0b2010 0x7f13fc0b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 >> v2:172.21.15.202:3300/0 conn(0x7f13fc153e50 msgr2=0x7f13fc0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13faffd700 1 --2- 172.21.15.47:0/2867498422 >> v2:172.21.15.202:3300/0 conn(0x7f13fc153e50 0x7f13fc0d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.703+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc15aed0 con 0x7f13fc153a80 2023-06-22T17:04:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 763095103 0 0) 0x7f13e4001470 con 0x7f13fc153a80 2023-06-22T17:04:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 >> v1:172.21.15.47:6789/0 conn(0x7f13fc153a80 legacy=0x7f13fc15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f13e8003b10 0x7f13e8003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 0x7f13e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc0b2010 0x7f13e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0d14c0 con 0x7f13e8004810 2023-06-22T17:04:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0b29d0 con 0x7f13e8003b10 2023-06-22T17:04:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e800bb40 con 0x7f13fc0b2010 2023-06-22T17:04:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13fb7fe700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 0x7f13e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13fbfff700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f13e8003b10 0x7f13e8003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f1400bc1700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc0b2010 0x7f13e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13fb7fe700 1 -- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc0b2010 msgr2=0x7f13e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13fb7fe700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc0b2010 0x7f13e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13fb7fe700 1 -- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f13e8003b10 msgr2=0x7f13e8003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13fb7fe700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f13e8003b10 0x7f13e8003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13fb7fe700 1 -- 172.21.15.47:0/2867498422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e8003570 con 0x7f13e8004810 2023-06-22T17:04:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13fb7fe700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 0x7f13e8004c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009c00 tx=0x7f13ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9968728ecc70da server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec00e710 con 0x7f13e8004810 2023-06-22T17:04:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec00e870 con 0x7f13e8004810 2023-06-22T17:04:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.704+0000 7f13faffd700 1 -- 172.21.15.47:0/2867498422 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec00ec30 con 0x7f13e8004810 2023-06-22T17:04:13.711 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.705+0000 7f1402624700 1 -- 172.21.15.47:0/2867498422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 msgr2=0x7f13e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.705+0000 7f1402624700 1 --2- 172.21.15.47:0/2867498422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 0x7f13e8004c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009c00 tx=0x7f13ec0091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.705+0000 7f1400bc1700 1 -- 172.21.15.47:0/2867498422 reap_dead start 2023-06-22T17:04:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.705+0000 7f1402624700 1 -- 172.21.15.47:0/2867498422 shutdown_connections 2023-06-22T17:04:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.705+0000 7f1402624700 1 -- 172.21.15.47:0/2867498422 >> 172.21.15.47:0/2867498422 conn(0x7f13fc070290 msgr2=0x7f13fc070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.705+0000 7f1402624700 1 -- 172.21.15.47:0/2867498422 shutdown_connections 2023-06-22T17:04:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.705+0000 7f1402624700 1 -- 172.21.15.47:0/2867498422 wait complete. 2023-06-22T17:04:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.706+0000 7f1402624700 1 Processor -- start 2023-06-22T17:04:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.706+0000 7f1402624700 1 -- start start 2023-06-22T17:04:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f1402624700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 0x7f13fc1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f1402624700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f13fc16d960 0x7f13fc150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f1402624700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc15d2c0 0x7f13fc15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f1402624700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc1565b0 con 0x7f13e8004810 2023-06-22T17:04:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f1402624700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc15dc50 con 0x7f13fc16d960 2023-06-22T17:04:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f1402624700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc16ddd0 con 0x7f13fc15d2c0 2023-06-22T17:04:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f13fbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 0x7f13fc1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f13fbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 0x7f13fc1553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51400/0 (socket says 172.21.15.47:51400) 2023-06-22T17:04:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f13fbfff700 1 -- 172.21.15.47:0/2156539115 learned_addr learned my addr 172.21.15.47:0/2156539115 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f13fb7fe700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f13fc16d960 0x7f13fc150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f1400bc1700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc15d2c0 0x7f13fc15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f13fbfff700 1 -- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc15d2c0 msgr2=0x7f13fc15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:13.716 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f13fbfff700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc15d2c0 0x7f13fc15d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f13fbfff700 1 -- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f13fc16d960 msgr2=0x7f13fc150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f13fbfff700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f13fc16d960 0x7f13fc150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f13fbfff700 1 -- 172.21.15.47:0/2156539115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc1699f0 con 0x7f13e8004810 2023-06-22T17:04:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.707+0000 7f13fbfff700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 0x7f13fc1553c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f13e4007370 tx=0x7f13e4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.708+0000 7f13f3fff700 1 -- 172.21.15.47:0/2156539115 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e400f660 con 0x7f13e8004810 2023-06-22T17:04:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.708+0000 7f13f3fff700 1 -- 172.21.15.47:0/2156539115 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e400f7c0 con 0x7f13e8004810 2023-06-22T17:04:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.708+0000 7f1402624700 1 -- 172.21.15.47:0/2156539115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13fc1559a0 con 0x7f13e8004810 2023-06-22T17:04:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.708+0000 7f13f3fff700 1 -- 172.21.15.47:0/2156539115 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e400fbc0 con 0x7f13e8004810 2023-06-22T17:04:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.708+0000 7f1402624700 1 -- 172.21.15.47:0/2156539115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13fc155f60 con 0x7f13e8004810 2023-06-22T17:04:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.708+0000 7f1402624700 1 -- 172.21.15.47:0/2156539115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13fc156f10 con 0x7f13e8004810 2023-06-22T17:04:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.709+0000 7f13f3fff700 1 -- 172.21.15.47:0/2156539115 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f13e401e3b0 con 0x7f13e8004810 2023-06-22T17:04:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.709+0000 7f13f3fff700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f13dc066e90 0x7f13dc069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.710+0000 7f13f3fff700 1 -- 172.21.15.47:0/2156539115 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f13e4084060 con 0x7f13e8004810 2023-06-22T17:04:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.710+0000 7f13fb7fe700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f13dc066e90 0x7f13dc069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:13.719 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.710+0000 7f13fb7fe700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f13dc066e90 0x7f13dc069340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0091e0 tx=0x7f13ec01a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:13.719 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.714+0000 7f13f3fff700 1 -- 172.21.15.47:0/2156539115 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f13e40511f0 con 0x7f13e8004810 2023-06-22T17:04:13.952 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.951+0000 7f1402624700 1 -- 172.21.15.47:0/2156539115 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f13fc04c1b0 con 0x7f13e8004810 2023-06-22T17:04:13.957 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.957+0000 7f13f3fff700 1 -- 172.21.15.47:0/2156539115 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f13e401a020 con 0x7f13e8004810 2023-06-22T17:04:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.964+0000 7f1402624700 1 -- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f13dc066e90 msgr2=0x7f13dc069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:13.965 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.964+0000 7f1402624700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f13dc066e90 0x7f13dc069340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0091e0 tx=0x7f13ec01a000 comp rx=0 tx=0).stop 2023-06-22T17:04:13.966 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.964+0000 7f1402624700 1 -- 172.21.15.47:0/2156539115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 msgr2=0x7f13fc1553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:13.966 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.964+0000 7f1402624700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 0x7f13fc1553c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f13e4007370 tx=0x7f13e4001d80 comp rx=0 tx=0).stop 2023-06-22T17:04:13.967 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.965+0000 7f1402624700 1 -- 172.21.15.47:0/2156539115 shutdown_connections 2023-06-22T17:04:13.967 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.965+0000 7f1402624700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f13dc066e90 0x7f13dc069340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.968 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.965+0000 7f1402624700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f13fc15d2c0 0x7f13fc15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.968 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.965+0000 7f1402624700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e8004810 0x7f13fc1553c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.968 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.965+0000 7f1402624700 1 --2- 172.21.15.47:0/2156539115 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f13fc16d960 0x7f13fc150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:13.968 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.965+0000 7f1402624700 1 -- 172.21.15.47:0/2156539115 >> 172.21.15.47:0/2156539115 conn(0x7f13fc070290 msgr2=0x7f13fc15c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:13.969 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.969+0000 7f1402624700 1 -- 172.21.15.47:0/2156539115 shutdown_connections 2023-06-22T17:04:13.969 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:13.969+0000 7f1402624700 1 -- 172.21.15.47:0/2156539115 wait complete. 2023-06-22T17:04:13.976 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 3 and data pool 4 2023-06-22T17:04:13.991 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-22T17:04:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.108+0000 7ff18e89a700 1 Processor -- start 2023-06-22T17:04:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.109+0000 7ff18e89a700 1 -- start start 2023-06-22T17:04:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.109+0000 7ff18e89a700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7ff1881594b0 0x7ff1880ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.109+0000 7ff18e89a700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff1880ca730 0x7ff1880a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.109+0000 7ff18e89a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1880aadb0 con 0x7ff188159880 2023-06-22T17:04:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.109+0000 7ff18e89a700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1880aa1a0 con 0x7ff1881594b0 2023-06-22T17:04:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.109+0000 7ff18e89a700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1880aa480 con 0x7ff1880ca730 2023-06-22T17:04:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.109+0000 7ff18ce37700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff188159880 0x7ff188068420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57066/0 (socket says 172.21.15.47:57066) 2023-06-22T17:04:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.109+0000 7ff18ce37700 1 -- 172.21.15.47:0/3593042730 learned_addr learned my addr 172.21.15.47:0/3593042730 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff1877fe700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff1880ca730 0x7ff1880a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff187fff700 1 --2- 172.21.15.47:0/3593042730 >> v2:172.21.15.202:3300/0 conn(0x7ff1881594b0 0x7ff1880ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39968540 0 0) 0x7ff1880aadb0 con 0x7ff188159880 2023-06-22T17:04:14.121 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff170003570 con 0x7ff188159880 2023-06-22T17:04:14.122 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4013746626 0 0) 0x7ff170003570 con 0x7ff188159880 2023-06-22T17:04:14.122 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff1880ca730 msgr2=0x7ff1880a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:14.122 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff1880ca730 0x7ff1880a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.123 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 >> v2:172.21.15.202:3300/0 conn(0x7ff1881594b0 msgr2=0x7ff1880ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:14.123 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 --2- 172.21.15.47:0/3593042730 >> v2:172.21.15.202:3300/0 conn(0x7ff1881594b0 0x7ff1880ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.123 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1880aa760 con 0x7ff188159880 2023-06-22T17:04:14.123 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 763095103 0 0) 0x7ff1800017f0 con 0x7ff188159880 2023-06-22T17:04:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 >> v1:172.21.15.47:6789/0 conn(0x7ff188159880 legacy=0x7ff188068420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff1880ca730 0x7ff170003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 0x7ff170004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7ff170004d50 0x7ff1700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1880aa1a0 con 0x7ff170003fe0 2023-06-22T17:04:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1880aa480 con 0x7ff170004d50 2023-06-22T17:04:14.124 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17000ba90 con 0x7ff1880ca730 2023-06-22T17:04:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff1877fe700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 0x7ff170004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff187fff700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff1880ca730 0x7ff170003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff18ce37700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7ff170004d50 0x7ff1700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff1877fe700 1 -- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff1880ca730 msgr2=0x7ff170003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff1877fe700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff1880ca730 0x7ff170003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.125 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff1877fe700 1 -- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7ff170004d50 msgr2=0x7ff1700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff1877fe700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7ff170004d50 0x7ff1700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.110+0000 7ff1877fe700 1 -- 172.21.15.47:0/3593042730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff170003570 con 0x7ff170003fe0 2023-06-22T17:04:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff1877fe700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 0x7ff170004810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff1880cab00 tx=0x7ff17c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a8e768853e1a19 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00e6b0 con 0x7ff170003fe0 2023-06-22T17:04:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c00e810 con 0x7ff170003fe0 2023-06-22T17:04:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff186ffd700 1 -- 172.21.15.47:0/3593042730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00ebd0 con 0x7ff170003fe0 2023-06-22T17:04:14.126 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff18e89a700 1 -- 172.21.15.47:0/3593042730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 msgr2=0x7ff170004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.127 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff18e89a700 1 --2- 172.21.15.47:0/3593042730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 0x7ff170004810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff1880cab00 tx=0x7ff17c0091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.127 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff18ce37700 1 -- 172.21.15.47:0/3593042730 reap_dead start 2023-06-22T17:04:14.127 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff18e89a700 1 -- 172.21.15.47:0/3593042730 shutdown_connections 2023-06-22T17:04:14.127 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff18e89a700 1 -- 172.21.15.47:0/3593042730 >> 172.21.15.47:0/3593042730 conn(0x7ff1880c9fc0 msgr2=0x7ff18806d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:14.127 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff18e89a700 1 -- 172.21.15.47:0/3593042730 shutdown_connections 2023-06-22T17:04:14.127 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.111+0000 7ff18e89a700 1 -- 172.21.15.47:0/3593042730 wait complete. 2023-06-22T17:04:14.127 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.112+0000 7ff18e89a700 1 Processor -- start 2023-06-22T17:04:14.128 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.113+0000 7ff18e89a700 1 -- start start 2023-06-22T17:04:14.128 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.113+0000 7ff18e89a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 0x7ff1881565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.128 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.113+0000 7ff18e89a700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff18815c880 0x7ff1880c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.128 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.113+0000 7ff18e89a700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7ff1880c2f90 0x7ff188169fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.129 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.113+0000 7ff18e89a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1880c3430 con 0x7ff170003fe0 2023-06-22T17:04:14.129 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.113+0000 7ff18e89a700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1880af0d0 con 0x7ff1880c2f90 2023-06-22T17:04:14.129 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.113+0000 7ff18e89a700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18816af40 con 0x7ff18815c880 2023-06-22T17:04:14.129 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.113+0000 7ff187fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 0x7ff1881565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.130 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.113+0000 7ff187fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 0x7ff1881565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51432/0 (socket says 172.21.15.47:51432) 2023-06-22T17:04:14.130 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.113+0000 7ff187fff700 1 -- 172.21.15.47:0/3376889326 learned_addr learned my addr 172.21.15.47:0/3376889326 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:14.130 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff1877fe700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff18815c880 0x7ff1880c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.130 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff18ce37700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7ff1880c2f90 0x7ff188169fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.131 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff187fff700 1 -- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff18815c880 msgr2=0x7ff1880c2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.131 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff187fff700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff18815c880 0x7ff1880c2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.131 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff187fff700 1 -- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7ff1880c2f90 msgr2=0x7ff188169fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.131 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff187fff700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7ff1880c2f90 0x7ff188169fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.131 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff187fff700 1 -- 172.21.15.47:0/3376889326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18815c3f0 con 0x7ff170003fe0 2023-06-22T17:04:14.132 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff187fff700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 0x7ff1881565d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff178011d40 tx=0x7ff1780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:14.132 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff16ffff700 1 -- 172.21.15.47:0/3376889326 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17800fac0 con 0x7ff170003fe0 2023-06-22T17:04:14.132 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff16ffff700 1 -- 172.21.15.47:0/3376889326 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17800fc20 con 0x7ff170003fe0 2023-06-22T17:04:14.132 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff16ffff700 1 -- 172.21.15.47:0/3376889326 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178014440 con 0x7ff170003fe0 2023-06-22T17:04:14.132 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff18e89a700 1 -- 172.21.15.47:0/3376889326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18816ac60 con 0x7ff170003fe0 2023-06-22T17:04:14.133 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.114+0000 7ff18e89a700 1 -- 172.21.15.47:0/3376889326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18815c080 con 0x7ff170003fe0 2023-06-22T17:04:14.133 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.115+0000 7ff18e89a700 1 -- 172.21.15.47:0/3376889326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18804c1b0 con 0x7ff170003fe0 2023-06-22T17:04:14.133 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.115+0000 7ff16ffff700 1 -- 172.21.15.47:0/3376889326 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7ff178014a80 con 0x7ff170003fe0 2023-06-22T17:04:14.133 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.116+0000 7ff16ffff700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7ff174066e90 0x7ff174069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.133 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.116+0000 7ff16ffff700 1 -- 172.21.15.47:0/3376889326 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff178018030 con 0x7ff170003fe0 2023-06-22T17:04:14.133 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.116+0000 7ff1877fe700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7ff174066e90 0x7ff174069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.133 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.117+0000 7ff1877fe700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7ff174066e90 0x7ff174069340 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0091e0 tx=0x7ff17c01a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:14.134 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.121+0000 7ff16ffff700 1 -- 172.21.15.47:0/3376889326 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7ff1780524d0 con 0x7ff170003fe0 2023-06-22T17:04:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.316+0000 7ff18e89a700 1 -- 172.21.15.47:0/3376889326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1880aaa40 con 0x7ff170003fe0 2023-06-22T17:04:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.317+0000 7ff16ffff700 1 -- 172.21.15.47:0/3376889326 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16879 (secure 0 0 0) 0x7ff178055af0 con 0x7ff170003fe0 2023-06-22T17:04:14.318 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:04:14.318 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"fsid":"61e931aa-8dfe-4361-8a29-1d30432b403b","created":"2023-06-22T17:03:48.391484+0000","modified":"2023-06-22T17:04:13.952651+0000","last_up_change":"2023-06-22T17:03:59.369744+0000","last_in_change":"2023-06-22T17:03:50.761615+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-22T17:04:00.277500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-22T17:04:06.740390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-22T17:04:11.598166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-22T17:04:12.931654+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191ca9cf-dc97-4325-b63b-a3e6a66b71b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":87966},{"type":"v1","addr":"172.21.15.47:6809","nonce":87966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":87966},{"type":"v1","addr":"172.21.15.47:6811","nonce":87966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":87966},{"type":"v1","addr":"172.21.15.47:6815","nonce":87966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":87966},{"type":"v1","addr":"172.21.15.47:6813","nonce":87966}]},"public_addr":"172.21.15.47:6809/87966","cluster_addr":"172.21.15.47:6811/87966","heartbeat_back_addr":"172.21.15.47:6815/87966","heartbeat_front_addr":"172.21.15.47:6813/87966","state":["exists","up"]},{"osd":1,"uuid":"d426e8cd-b1e6-40a3-9e65-cf52fac95a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":87978},{"type":"v1","addr":"172.21.15.47:6825","nonce":87978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":87978},{"type":"v1","addr":"172.21.15.47:6827","nonce":87978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":87978},{"type":"v1","addr":"172.21.15.47:6831","nonce":87978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":87978},{"type":"v1","addr":"172.21.15.47:6829","nonce":87978}]},"public_addr":"172.21.15.47:6825/87978","cluster_addr":"172.21.15.47:6827/87978","heartbeat_back_addr":"172.21.15.47:6831/87978","heartbeat_front_addr":"172.21.15.47:6829/87978","state":["exists","up"]},{"osd":2,"uuid":"487a8fca-1da6-4cb3-a73c-abc6e9e2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":87967},{"type":"v1","addr":"172.21.15.47:6801","nonce":87967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":87967},{"type":"v1","addr":"172.21.15.47:6803","nonce":87967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":87967},{"type":"v1","addr":"172.21.15.47:6807","nonce":87967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":87967},{"type":"v1","addr":"172.21.15.47:6805","nonce":87967}]},"public_addr":"172.21.15.47:6801/87967","cluster_addr":"172.21.15.47:6803/87967","heartbeat_back_addr":"172.21.15.47:6807/87967","heartbeat_front_addr":"172.21.15.47:6805/87967","state":["exists","up"]},{"osd":3,"uuid":"7089499f-dc69-4d6b-a6c3-5b30404ab1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":87968},{"type":"v1","addr":"172.21.15.47:6817","nonce":87968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":87968},{"type":"v1","addr":"172.21.15.47:6819","nonce":87968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":87968},{"type":"v1","addr":"172.21.15.47:6823","nonce":87968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":87968},{"type":"v1","addr":"172.21.15.47:6821","nonce":87968}]},"public_addr":"172.21.15.47:6817/87968","cluster_addr":"172.21.15.47:6819/87968","heartbeat_back_addr":"172.21.15.47:6823/87968","heartbeat_front_addr":"172.21.15.47:6821/87968","state":["exists","up"]},{"osd":4,"uuid":"f8b34dee-7945-42fd-9e58-98e5fe196b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6800","nonce":88201},{"type":"v1","addr":"172.21.15.202:6801","nonce":88201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6802","nonce":88201},{"type":"v1","addr":"172.21.15.202:6803","nonce":88201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6806","nonce":88201},{"type":"v1","addr":"172.21.15.202:6807","nonce":88201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6804","nonce":88201},{"type":"v1","addr":"172.21.15.202:6805","nonce":88201}]},"public_addr":"172.21.15.202:6801/88201","cluster_addr":"172.21.15.202:6803/88201","heartbeat_back_addr":"172.21.15.202:6807/88201","heartbeat_front_addr":"172.21.15.202:6805/88201","state":["exists","up"]},{"osd":5,"uuid":"9583b379-49e1-4b6c-adf5-6f19322cc76c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6808","nonce":88199},{"type":"v1","addr":"172.21.15.202:6809","nonce":88199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6810","nonce":88199},{"type":"v1","addr":"172.21.15.202:6812","nonce":88199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6818","nonce":88199},{"type":"v1","addr":"172.21.15.202:6820","nonce":88199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6814","nonce":88199},{"type":"v1","addr":"172.21.15.202:6816","nonce":88199}]},"public_addr":"172.21.15.202:6809/88199","cluster_addr":"172.21.15.202:6812/88199","heartbeat_back_addr":"172.21.15.202:6820/88199","heartbeat_front_addr":"172.21.15.202:6816/88199","state":["exists","up"]},{"osd":6,"uuid":"d11633b5-5a36-441b-93c7-7c0b434f0dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6811","nonce":88200},{"type":"v1","addr":"172.21.15.202:6813","nonce":88200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6815","nonce":88200},{"type":"v1","addr":"172.21.15.202:6817","nonce":88200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6822","nonce":88200},{"type":"v1","addr":"172.21.15.202:6823","nonce":88200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6819","nonce":88200},{"type":"v1","addr":"172.21.15.202:6821","nonce":88200}]},"public_addr":"172.21.15.202:6813/88200","cluster_addr":"172.21.15.202:6817/88200","heartbeat_back_addr":"172.21.15.202:6823/88200","heartbeat_front_addr":"172.21.15.202:6821/88200","state":["exists","up"]},{"osd":7,"uuid":"edd9bf42-9566-4d07-902b-08cf16cb42a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6824","nonce":88230},{"type":"v1","addr":"172.21.15.202:6825","nonce":88230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6826","nonce":88230},{"type":"v1","addr":"172.21.15.202:6827","nonce":88230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6830","nonce":88230},{"type":"v1","addr":"172.21.15.202:6831","nonce":88230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6828","nonce":88230},{"type":"v1","addr":"172.21.15.202:6829","nonce":88230}]},"public_addr":"172.21.15.202:6825/88230","cluster_addr":"172.21.15.202:6827/88230","heartbeat_back_addr":"172.21.15.202:6831/88230","heartbeat_front_addr":"172.21.15.202:6829/88230","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.975830+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.942848+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.958568+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.972986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.946508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.938793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.960236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.931356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T17:04:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.319+0000 7ff18e89a700 1 -- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7ff174066e90 msgr2=0x7ff174069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.319+0000 7ff18e89a700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7ff174066e90 0x7ff174069340 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0091e0 tx=0x7ff17c01a000 comp rx=0 tx=0).stop 2023-06-22T17:04:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.319+0000 7ff18e89a700 1 -- 172.21.15.47:0/3376889326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 msgr2=0x7ff1881565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.319+0000 7ff18e89a700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 0x7ff1881565d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff178011d40 tx=0x7ff1780091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.319+0000 7ff18e89a700 1 -- 172.21.15.47:0/3376889326 shutdown_connections 2023-06-22T17:04:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.319+0000 7ff18e89a700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7ff174066e90 0x7ff174069340 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.319+0000 7ff18e89a700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7ff1880c2f90 0x7ff188169fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.319+0000 7ff18e89a700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff170003fe0 0x7ff1881565d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.319+0000 7ff18e89a700 1 --2- 172.21.15.47:0/3376889326 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7ff18815c880 0x7ff1880c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.320+0000 7ff18e89a700 1 -- 172.21.15.47:0/3376889326 >> 172.21.15.47:0/3376889326 conn(0x7ff1880c9fc0 msgr2=0x7ff1881573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.320+0000 7ff18e89a700 1 -- 172.21.15.47:0/3376889326 shutdown_connections 2023-06-22T17:04:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.320+0000 7ff18e89a700 1 -- 172.21.15.47:0/3376889326 wait complete. 2023-06-22T17:04:14.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-22T17:04:14.401 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-22T17:04:14.404 DEBUG:teuthology.orchestra.run.smithi202:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-22T17:04:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.451+0000 7f6487413700 1 Processor -- start 2023-06-22T17:04:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.451+0000 7f6487413700 1 -- start start 2023-06-22T17:04:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.452+0000 7f6487413700 1 --2- >> v2:172.21.15.202:3300/0 conn(0x7f64800ad470 0x7f64800abf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.452+0000 7f6487413700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648015a280 con 0x7f64801508d0 2023-06-22T17:04:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.452+0000 7f6487413700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64800afa00 con 0x7f64800af570 2023-06-22T17:04:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.452+0000 7f6487413700 1 -- --> v2:172.21.15.202:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64800afce0 con 0x7f64800ad470 2023-06-22T17:04:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.452+0000 7f64851af700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f64801508d0 0x7f64800aee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57078/0 (socket says 172.21.15.47:57078) 2023-06-22T17:04:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.452+0000 7f64851af700 1 -- 172.21.15.47:0/3280380887 learned_addr learned my addr 172.21.15.47:0/3280380887 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.452+0000 7f64859b0700 1 --2- 172.21.15.47:0/3280380887 >> v2:172.21.15.202:3300/0 conn(0x7f64800ad470 0x7f64800abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f647ffff700 1 -- 172.21.15.47:0/3280380887 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946883269 0 0) 0x7f648015a280 con 0x7f64801508d0 2023-06-22T17:04:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f647ffff700 1 -- 172.21.15.47:0/3280380887 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6468003570 con 0x7f64801508d0 2023-06-22T17:04:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f647ffff700 1 -- 172.21.15.47:0/3280380887 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246561189 0 0) 0x7f64800afa00 con 0x7f64800af570 2023-06-22T17:04:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f647ffff700 1 -- 172.21.15.47:0/3280380887 --> v1:172.21.15.202:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f648015a280 con 0x7f64800af570 2023-06-22T17:04:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f64859b0700 1 -- 172.21.15.47:0/3280380887 >> v1:172.21.15.202:6789/0 conn(0x7f64800af570 legacy=0x7f64800acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f64859b0700 1 -- 172.21.15.47:0/3280380887 >> v1:172.21.15.47:6789/0 conn(0x7f64801508d0 legacy=0x7f64800aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f64859b0700 1 -- 172.21.15.47:0/3280380887 --> v2:172.21.15.202:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64800affc0 con 0x7f64800ad470 2023-06-22T17:04:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f64859b0700 1 --2- 172.21.15.47:0/3280380887 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f64800ad470 0x7f64800abf00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64780098e0 tx=0x7f64780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8bc00cb9affeeb server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f647ffff700 1 -- 172.21.15.47:0/3280380887 <== mon.1 v2:172.21.15.202:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478011630 con 0x7f64800ad470 2023-06-22T17:04:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f647ffff700 1 -- 172.21.15.47:0/3280380887 <== mon.1 v2:172.21.15.202:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6478011790 con 0x7f64800ad470 2023-06-22T17:04:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.453+0000 7f647ffff700 1 -- 172.21.15.47:0/3280380887 <== mon.1 v2:172.21.15.202:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478011b30 con 0x7f64800ad470 2023-06-22T17:04:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.454+0000 7f6487413700 1 -- 172.21.15.47:0/3280380887 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f64800ad470 msgr2=0x7f64800abf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.454+0000 7f6487413700 1 --2- 172.21.15.47:0/3280380887 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f64800ad470 0x7f64800abf00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64780098e0 tx=0x7f64780091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.454+0000 7f6487413700 1 -- 172.21.15.47:0/3280380887 shutdown_connections 2023-06-22T17:04:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.454+0000 7f6487413700 1 --2- 172.21.15.47:0/3280380887 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f64800ad470 0x7f64800abf00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.454+0000 7f6487413700 1 -- 172.21.15.47:0/3280380887 >> 172.21.15.47:0/3280380887 conn(0x7f6480067bf0 msgr2=0x7f6480087f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.454+0000 7f6487413700 1 -- 172.21.15.47:0/3280380887 shutdown_connections 2023-06-22T17:04:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.454+0000 7f6487413700 1 -- 172.21.15.47:0/3280380887 wait complete. 2023-06-22T17:04:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.455+0000 7f6487413700 1 Processor -- start 2023-06-22T17:04:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.455+0000 7f6487413700 1 -- start start 2023-06-22T17:04:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f6487413700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f64800ad470 0x7f64801522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f6487413700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f64800af570 0x7f64801532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.459 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f6487413700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64801508d0 0x7f648014dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f6487413700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6480153de0 con 0x7f64801508d0 2023-06-22T17:04:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f6487413700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64800c81f0 con 0x7f64800ad470 2023-06-22T17:04:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f6487413700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6480153b00 con 0x7f64800af570 2023-06-22T17:04:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64859b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64801508d0 0x7f648014dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64859b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64801508d0 0x7f648014dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51438/0 (socket says 172.21.15.47:51438) 2023-06-22T17:04:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64859b0700 1 -- 172.21.15.47:0/3147329217 learned_addr learned my addr 172.21.15.47:0/3147329217 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64851af700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f64800ad470 0x7f64801522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64849ae700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f64800af570 0x7f64801532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64859b0700 1 -- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f64800af570 msgr2=0x7f64801532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64859b0700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f64800af570 0x7f64801532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64859b0700 1 -- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f64800ad470 msgr2=0x7f64801522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64859b0700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f64800ad470 0x7f64801522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64859b0700 1 -- 172.21.15.47:0/3147329217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6478009420 con 0x7f64801508d0 2023-06-22T17:04:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64851af700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f64800ad470 0x7f64801522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T17:04:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f64859b0700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64801508d0 0x7f648014dfc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6478013000 tx=0x7f6478017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f647d7fa700 1 -- 172.21.15.47:0/3147329217 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647801f070 con 0x7f64801508d0 2023-06-22T17:04:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f647d7fa700 1 -- 172.21.15.47:0/3147329217 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6478011900 con 0x7f64801508d0 2023-06-22T17:04:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f6487413700 1 -- 172.21.15.47:0/3147329217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64800c7ba0 con 0x7f64801508d0 2023-06-22T17:04:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f647d7fa700 1 -- 172.21.15.47:0/3147329217 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478025420 con 0x7f64801508d0 2023-06-22T17:04:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.456+0000 7f6487413700 1 -- 172.21.15.47:0/3147329217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6480153810 con 0x7f64801508d0 2023-06-22T17:04:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.457+0000 7f6487413700 1 -- 172.21.15.47:0/3147329217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64800c7a00 con 0x7f64801508d0 2023-06-22T17:04:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.458+0000 7f647d7fa700 1 -- 172.21.15.47:0/3147329217 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f6478023070 con 0x7f64801508d0 2023-06-22T17:04:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.458+0000 7f647d7fa700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f646c066e90 0x7f646c069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.458+0000 7f647d7fa700 1 -- 172.21.15.47:0/3147329217 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6478085ae0 con 0x7f64801508d0 2023-06-22T17:04:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.458+0000 7f64851af700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f646c066e90 0x7f646c069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.459+0000 7f64851af700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f646c066e90 0x7f646c069340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6470007d60 tx=0x7f6470009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:14.465 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.462+0000 7f647d7fa700 1 -- 172.21.15.47:0/3147329217 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f64800c7a00 con 0x7f64801508d0 2023-06-22T17:04:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.659+0000 7f6487413700 1 -- 172.21.15.47:0/3147329217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64800c7a00 con 0x7f64801508d0 2023-06-22T17:04:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.659+0000 7f647d7fa700 1 -- 172.21.15.47:0/3147329217 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f64800c7a00 con 0x7f64801508d0 2023-06-22T17:04:14.661 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:04:14.661 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"fsid":"61e931aa-8dfe-4361-8a29-1d30432b403b","created":"2023-06-22T17:03:48.391484+0000","modified":"2023-06-22T17:04:13.952651+0000","last_up_change":"2023-06-22T17:03:59.369744+0000","last_in_change":"2023-06-22T17:03:50.761615+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-22T17:04:00.277500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-22T17:04:06.740390+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-22T17:04:11.598166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-22T17:04:12.931654+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"191ca9cf-dc97-4325-b63b-a3e6a66b71b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":87966},{"type":"v1","addr":"172.21.15.47:6809","nonce":87966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":87966},{"type":"v1","addr":"172.21.15.47:6811","nonce":87966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":87966},{"type":"v1","addr":"172.21.15.47:6815","nonce":87966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":87966},{"type":"v1","addr":"172.21.15.47:6813","nonce":87966}]},"public_addr":"172.21.15.47:6809/87966","cluster_addr":"172.21.15.47:6811/87966","heartbeat_back_addr":"172.21.15.47:6815/87966","heartbeat_front_addr":"172.21.15.47:6813/87966","state":["exists","up"]},{"osd":1,"uuid":"d426e8cd-b1e6-40a3-9e65-cf52fac95a21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":87978},{"type":"v1","addr":"172.21.15.47:6825","nonce":87978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":87978},{"type":"v1","addr":"172.21.15.47:6827","nonce":87978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":87978},{"type":"v1","addr":"172.21.15.47:6831","nonce":87978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":87978},{"type":"v1","addr":"172.21.15.47:6829","nonce":87978}]},"public_addr":"172.21.15.47:6825/87978","cluster_addr":"172.21.15.47:6827/87978","heartbeat_back_addr":"172.21.15.47:6831/87978","heartbeat_front_addr":"172.21.15.47:6829/87978","state":["exists","up"]},{"osd":2,"uuid":"487a8fca-1da6-4cb3-a73c-abc6e9e2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":87967},{"type":"v1","addr":"172.21.15.47:6801","nonce":87967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":87967},{"type":"v1","addr":"172.21.15.47:6803","nonce":87967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":87967},{"type":"v1","addr":"172.21.15.47:6807","nonce":87967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":87967},{"type":"v1","addr":"172.21.15.47:6805","nonce":87967}]},"public_addr":"172.21.15.47:6801/87967","cluster_addr":"172.21.15.47:6803/87967","heartbeat_back_addr":"172.21.15.47:6807/87967","heartbeat_front_addr":"172.21.15.47:6805/87967","state":["exists","up"]},{"osd":3,"uuid":"7089499f-dc69-4d6b-a6c3-5b30404ab1bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":87968},{"type":"v1","addr":"172.21.15.47:6817","nonce":87968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":87968},{"type":"v1","addr":"172.21.15.47:6819","nonce":87968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":87968},{"type":"v1","addr":"172.21.15.47:6823","nonce":87968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":87968},{"type":"v1","addr":"172.21.15.47:6821","nonce":87968}]},"public_addr":"172.21.15.47:6817/87968","cluster_addr":"172.21.15.47:6819/87968","heartbeat_back_addr":"172.21.15.47:6823/87968","heartbeat_front_addr":"172.21.15.47:6821/87968","state":["exists","up"]},{"osd":4,"uuid":"f8b34dee-7945-42fd-9e58-98e5fe196b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6800","nonce":88201},{"type":"v1","addr":"172.21.15.202:6801","nonce":88201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6802","nonce":88201},{"type":"v1","addr":"172.21.15.202:6803","nonce":88201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6806","nonce":88201},{"type":"v1","addr":"172.21.15.202:6807","nonce":88201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6804","nonce":88201},{"type":"v1","addr":"172.21.15.202:6805","nonce":88201}]},"public_addr":"172.21.15.202:6801/88201","cluster_addr":"172.21.15.202:6803/88201","heartbeat_back_addr":"172.21.15.202:6807/88201","heartbeat_front_addr":"172.21.15.202:6805/88201","state":["exists","up"]},{"osd":5,"uuid":"9583b379-49e1-4b6c-adf5-6f19322cc76c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6808","nonce":88199},{"type":"v1","addr":"172.21.15.202:6809","nonce":88199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6810","nonce":88199},{"type":"v1","addr":"172.21.15.202:6812","nonce":88199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6818","nonce":88199},{"type":"v1","addr":"172.21.15.202:6820","nonce":88199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6814","nonce":88199},{"type":"v1","addr":"172.21.15.202:6816","nonce":88199}]},"public_addr":"172.21.15.202:6809/88199","cluster_addr":"172.21.15.202:6812/88199","heartbeat_back_addr":"172.21.15.202:6820/88199","heartbeat_front_addr":"172.21.15.202:6816/88199","state":["exists","up"]},{"osd":6,"uuid":"d11633b5-5a36-441b-93c7-7c0b434f0dbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6811","nonce":88200},{"type":"v1","addr":"172.21.15.202:6813","nonce":88200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6815","nonce":88200},{"type":"v1","addr":"172.21.15.202:6817","nonce":88200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6822","nonce":88200},{"type":"v1","addr":"172.21.15.202:6823","nonce":88200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6819","nonce":88200},{"type":"v1","addr":"172.21.15.202:6821","nonce":88200}]},"public_addr":"172.21.15.202:6813/88200","cluster_addr":"172.21.15.202:6817/88200","heartbeat_back_addr":"172.21.15.202:6823/88200","heartbeat_front_addr":"172.21.15.202:6821/88200","state":["exists","up"]},{"osd":7,"uuid":"edd9bf42-9566-4d07-902b-08cf16cb42a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6824","nonce":88230},{"type":"v1","addr":"172.21.15.202:6825","nonce":88230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6826","nonce":88230},{"type":"v1","addr":"172.21.15.202:6827","nonce":88230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6830","nonce":88230},{"type":"v1","addr":"172.21.15.202:6831","nonce":88230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.202:6828","nonce":88230},{"type":"v1","addr":"172.21.15.202:6829","nonce":88230}]},"public_addr":"172.21.15.202:6825/88230","cluster_addr":"172.21.15.202:6827/88230","heartbeat_back_addr":"172.21.15.202:6831/88230","heartbeat_front_addr":"172.21.15.202:6829/88230","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.975830+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.942848+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.958568+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.972986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.946508+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.938793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.960236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T17:03:55.931356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T17:04:14.662 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 -- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f646c066e90 msgr2=0x7f646c069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.662 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f646c066e90 0x7f646c069340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6470007d60 tx=0x7f6470009040 comp rx=0 tx=0).stop 2023-06-22T17:04:14.662 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 -- 172.21.15.47:0/3147329217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64801508d0 msgr2=0x7f648014dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.662 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64801508d0 0x7f648014dfc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6478013000 tx=0x7f6478017000 comp rx=0 tx=0).stop 2023-06-22T17:04:14.663 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 -- 172.21.15.47:0/3147329217 shutdown_connections 2023-06-22T17:04:14.663 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f646c066e90 0x7f646c069340 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.663 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64801508d0 0x7f648014dfc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.663 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f64800ad470 0x7f64801522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.663 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 --2- 172.21.15.47:0/3147329217 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f64800af570 0x7f64801532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.663 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 -- 172.21.15.47:0/3147329217 >> 172.21.15.47:0/3147329217 conn(0x7f6480067bf0 msgr2=0x7f648014ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:14.663 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 -- 172.21.15.47:0/3147329217 shutdown_connections 2023-06-22T17:04:14.663 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.661+0000 7f6487413700 1 -- 172.21.15.47:0/3147329217 wait complete. 2023-06-22T17:04:14.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-22T17:04:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.796+0000 7f68ecdca700 1 Processor -- start 2023-06-22T17:04:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.796+0000 7f68ecdca700 1 -- start start 2023-06-22T17:04:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.797+0000 7f68ecdca700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f68e8153e50 0x7f68e80d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.797+0000 7f68ecdca700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e80cedd0 con 0x7f68e80b2010 2023-06-22T17:04:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.797+0000 7f68ecdca700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e815aed0 con 0x7f68e8153a80 2023-06-22T17:04:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.797+0000 7f68ecdca700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80d14c0 con 0x7f68e8153e50 2023-06-22T17:04:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.797+0000 7f68e6d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f68e80b2010 0x7f68e80cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57082/0 (socket says 172.21.15.47:57082) 2023-06-22T17:04:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.797+0000 7f68e6d9d700 1 -- 172.21.15.47:0/1152217360 learned_addr learned my addr 172.21.15.47:0/1152217360 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e5d9b700 1 --2- 172.21.15.47:0/1152217360 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f68e8153e50 0x7f68e80d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e559a700 1 -- 172.21.15.47:0/1152217360 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325922243 0 0) 0x7f68e80cedd0 con 0x7f68e80b2010 2023-06-22T17:04:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e559a700 1 -- 172.21.15.47:0/1152217360 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d0003570 con 0x7f68e80b2010 2023-06-22T17:04:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e559a700 1 -- 172.21.15.47:0/1152217360 <== mon.1 v1:172.21.15.202:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437145517 0 0) 0x7f68e815aed0 con 0x7f68e8153a80 2023-06-22T17:04:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e559a700 1 -- 172.21.15.47:0/1152217360 --> v1:172.21.15.202:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e80cedd0 con 0x7f68e8153a80 2023-06-22T17:04:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e5d9b700 1 -- 172.21.15.47:0/1152217360 >> v1:172.21.15.202:6789/0 conn(0x7f68e8153a80 legacy=0x7f68e815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e559a700 1 -- 172.21.15.47:0/1152217360 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4179741151 0 0) 0x7f68d0003570 con 0x7f68e80b2010 2023-06-22T17:04:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e5d9b700 1 -- 172.21.15.47:0/1152217360 >> v1:172.21.15.47:6789/0 conn(0x7f68e80b2010 legacy=0x7f68e80cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e5d9b700 1 -- 172.21.15.47:0/1152217360 --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e80ceac0 con 0x7f68e8153e50 2023-06-22T17:04:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e5d9b700 1 --2- 172.21.15.47:0/1152217360 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f68e8153e50 0x7f68e80d0590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009910 tx=0x7f68dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d870137aea61010e server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e559a700 1 -- 172.21.15.47:0/1152217360 <== mon.2 v2:172.21.15.202:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00017f0 con 0x7f68e8153e50 2023-06-22T17:04:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e559a700 1 -- 172.21.15.47:0/1152217360 <== mon.2 v2:172.21.15.202:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc011420 con 0x7f68e8153e50 2023-06-22T17:04:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.798+0000 7f68e559a700 1 -- 172.21.15.47:0/1152217360 <== mon.2 v2:172.21.15.202:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc0117c0 con 0x7f68e8153e50 2023-06-22T17:04:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.799+0000 7f68ecdca700 1 -- 172.21.15.47:0/1152217360 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f68e8153e50 msgr2=0x7f68e80d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.799+0000 7f68ecdca700 1 --2- 172.21.15.47:0/1152217360 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f68e8153e50 0x7f68e80d0590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009910 tx=0x7f68dc0091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.799+0000 7f68ecdca700 1 -- 172.21.15.47:0/1152217360 shutdown_connections 2023-06-22T17:04:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.799+0000 7f68ecdca700 1 --2- 172.21.15.47:0/1152217360 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f68e8153e50 0x7f68e80d0590 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.799+0000 7f68ecdca700 1 -- 172.21.15.47:0/1152217360 >> 172.21.15.47:0/1152217360 conn(0x7f68e8070290 msgr2=0x7f68e8070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.799+0000 7f68ecdca700 1 -- 172.21.15.47:0/1152217360 shutdown_connections 2023-06-22T17:04:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.799+0000 7f68ecdca700 1 -- 172.21.15.47:0/1152217360 wait complete. 2023-06-22T17:04:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.800+0000 7f68ecdca700 1 Processor -- start 2023-06-22T17:04:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.800+0000 7f68ecdca700 1 -- start start 2023-06-22T17:04:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68ecdca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68e80b2010 0x7f68e80cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68ecdca700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f68e8153a80 0x7f68e816a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68ecdca700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f68e8153e50 0x7f68e8154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68ecdca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80cc9c0 con 0x7f68e80b2010 2023-06-22T17:04:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68ecdca700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e816a8c0 con 0x7f68e8153e50 2023-06-22T17:04:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68ecdca700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e816aba0 con 0x7f68e8153a80 2023-06-22T17:04:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68e80b2010 0x7f68e80cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68e80b2010 0x7f68e80cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51448/0 (socket says 172.21.15.47:51448) 2023-06-22T17:04:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e659c700 1 -- 172.21.15.47:0/16919293 learned_addr learned my addr 172.21.15.47:0/16919293 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e6d9d700 1 --2- 172.21.15.47:0/16919293 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f68e8153e50 0x7f68e8154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e5d9b700 1 --2- 172.21.15.47:0/16919293 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f68e8153a80 0x7f68e816a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e659c700 1 -- 172.21.15.47:0/16919293 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f68e8153a80 msgr2=0x7f68e816a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e659c700 1 --2- 172.21.15.47:0/16919293 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f68e8153a80 0x7f68e816a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e659c700 1 -- 172.21.15.47:0/16919293 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f68e8153e50 msgr2=0x7f68e8154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e659c700 1 --2- 172.21.15.47:0/16919293 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f68e8153e50 0x7f68e8154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e659c700 1 -- 172.21.15.47:0/16919293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68dc009420 con 0x7f68e80b2010 2023-06-22T17:04:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.801+0000 7f68e659c700 1 --2- 172.21.15.47:0/16919293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68e80b2010 0x7f68e80cc480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68d800c580 tx=0x7f68d8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.802+0000 7f68d67fc700 1 -- 172.21.15.47:0/16919293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8010510 con 0x7f68e80b2010 2023-06-22T17:04:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.802+0000 7f68d67fc700 1 -- 172.21.15.47:0/16919293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d8010670 con 0x7f68e80b2010 2023-06-22T17:04:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.802+0000 7f68ecdca700 1 -- 172.21.15.47:0/16919293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e816ae80 con 0x7f68e80b2010 2023-06-22T17:04:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.802+0000 7f68d67fc700 1 -- 172.21.15.47:0/16919293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8010a30 con 0x7f68e80b2010 2023-06-22T17:04:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.802+0000 7f68ecdca700 1 -- 172.21.15.47:0/16919293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e8154610 con 0x7f68e80b2010 2023-06-22T17:04:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.802+0000 7f68ecdca700 1 -- 172.21.15.47:0/16919293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e8154980 con 0x7f68e80b2010 2023-06-22T17:04:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.803+0000 7f68d67fc700 1 -- 172.21.15.47:0/16919293 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f68d801b400 con 0x7f68e80b2010 2023-06-22T17:04:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.803+0000 7f68d67fc700 1 --2- 172.21.15.47:0/16919293 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f68c8066e90 0x7f68c8069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.803+0000 7f68d67fc700 1 -- 172.21.15.47:0/16919293 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f68d80840d0 con 0x7f68e80b2010 2023-06-22T17:04:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.804+0000 7f68e5d9b700 1 --2- 172.21.15.47:0/16919293 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f68c8066e90 0x7f68c8069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:14.809 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.804+0000 7f68e5d9b700 1 --2- 172.21.15.47:0/16919293 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f68c8066e90 0x7f68c8069340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0030c0 tx=0x7f68dc016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:14.809 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:14.807+0000 7f68d67fc700 1 -- 172.21.15.47:0/16919293 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f68d80512a0 con 0x7f68e80b2010 2023-06-22T17:04:15.053 INFO:teuthology.orchestra.run.smithi047.stderr:no valid command found; 1 closest matches: 2023-06-22T17:04:15.053 INFO:teuthology.orchestra.run.smithi047.stderr:fs set |refuse_client_session|max_xattr_size [--yes-i-really-mean-it] [--yes-i-really-really-mean-it] 2023-06-22T17:04:15.053 INFO:teuthology.orchestra.run.smithi047.stderr:Error EINVAL: invalid command 2023-06-22T17:04:15.056 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-06-22T17:04:15.057 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-22T17:04:15.058 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-22T17:04:15.180 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.179+0000 7f5911c25700 1 Processor -- start 2023-06-22T17:04:15.180 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.179+0000 7f5911c25700 1 -- start start 2023-06-22T17:04:15.181 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.180+0000 7f5911c25700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f590c0b07f0 0x7f590c1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:15.182 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.180+0000 7f5911c25700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c0cb7d0 con 0x7f590c0b23d0 2023-06-22T17:04:15.182 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.180+0000 7f5911c25700 1 -- --> v1:172.21.15.202:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c0c72b0 con 0x7f590c159310 2023-06-22T17:04:15.182 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.180+0000 7f5911c25700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c0b2e10 con 0x7f590c0b07f0 2023-06-22T17:04:15.182 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.180+0000 7f5911424700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f590c0b07f0 0x7f590c1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:15.182 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.180+0000 7f590bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f590c0b23d0 0x7f590c0b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57088/0 (socket says 172.21.15.47:57088) 2023-06-22T17:04:15.182 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.180+0000 7f590bfff700 1 -- 172.21.15.47:0/1047664572 learned_addr learned my addr 172.21.15.47:0/1047664572 (peer_addr_for_me v1:172.21.15.47:0/0) 2023-06-22T17:04:15.182 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.180+0000 7f5911424700 1 -- 172.21.15.47:0/1047664572 >> v1:172.21.15.202:6789/0 conn(0x7f590c159310 legacy=0x7f590c15dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-22T17:04:15.183 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f5911424700 1 -- 172.21.15.47:0/1047664572 >> v1:172.21.15.47:6789/0 conn(0x7f590c0b23d0 legacy=0x7f590c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:15.183 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f5911424700 1 -- 172.21.15.47:0/1047664572 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c0b2850 con 0x7f590c0b07f0 2023-06-22T17:04:15.183 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f590b7fe700 1 -- 172.21.15.47:0/1047664572 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796188172 0 0) 0x7f590c0cb7d0 con 0x7f590c0b23d0 2023-06-22T17:04:15.183 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f5911424700 1 --2- 172.21.15.47:0/1047664572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f590c0b07f0 0x7f590c1596e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f58f40098e0 tx=0x7f58f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9452dbd44673ac9d server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:15.183 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f590b7fe700 1 -- 172.21.15.47:0/1047664572 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4009e80 con 0x7f590c0b07f0 2023-06-22T17:04:15.183 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f590b7fe700 1 -- 172.21.15.47:0/1047664572 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f400b820 con 0x7f590c0b07f0 2023-06-22T17:04:15.183 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f590b7fe700 1 -- 172.21.15.47:0/1047664572 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400bba0 con 0x7f590c0b07f0 2023-06-22T17:04:15.186 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f5911c25700 1 -- 172.21.15.47:0/1047664572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f590c0b07f0 msgr2=0x7f590c1596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:15.186 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f5911c25700 1 --2- 172.21.15.47:0/1047664572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f590c0b07f0 0x7f590c1596e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f58f40098e0 tx=0x7f58f40091a0 comp rx=0 tx=0).stop 2023-06-22T17:04:15.187 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f5911c25700 1 -- 172.21.15.47:0/1047664572 shutdown_connections 2023-06-22T17:04:15.187 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f5911c25700 1 --2- 172.21.15.47:0/1047664572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f590c0b07f0 0x7f590c1596e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:15.187 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f5911c25700 1 -- 172.21.15.47:0/1047664572 >> 172.21.15.47:0/1047664572 conn(0x7f590c070290 msgr2=0x7f590c070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T17:04:15.187 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f5911c25700 1 -- 172.21.15.47:0/1047664572 shutdown_connections 2023-06-22T17:04:15.187 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.181+0000 7f5911c25700 1 -- 172.21.15.47:0/1047664572 wait complete. 2023-06-22T17:04:15.187 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.182+0000 7f5911c25700 1 Processor -- start 2023-06-22T17:04:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.183+0000 7f5911c25700 1 -- start start 2023-06-22T17:04:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.183+0000 7f5911c25700 1 --2- >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f590c0b07f0 0x7f590c157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.183+0000 7f5911c25700 1 --2- >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f590c159310 0x7f590c0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.183+0000 7f5911c25700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f590c157950 0x7f590c0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.183+0000 7f5911c25700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c16d750 con 0x7f590c157950 2023-06-22T17:04:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.183+0000 7f5911c25700 1 -- --> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c16cb40 con 0x7f590c159310 2023-06-22T17:04:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.183+0000 7f5911c25700 1 -- --> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c16ce20 con 0x7f590c0b07f0 2023-06-22T17:04:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.183+0000 7f5911424700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f590c157950 0x7f590c0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.183+0000 7f5911424700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f590c157950 0x7f590c0b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51468/0 (socket says 172.21.15.47:51468) 2023-06-22T17:04:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.183+0000 7f5911424700 1 -- 172.21.15.47:0/2553668016 learned_addr learned my addr 172.21.15.47:0/2553668016 (peer_addr_for_me v2:172.21.15.47:0/0) 2023-06-22T17:04:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f590bfff700 1 --2- 172.21.15.47:0/2553668016 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f590c159310 0x7f590c0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5910c23700 1 --2- 172.21.15.47:0/2553668016 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f590c0b07f0 0x7f590c157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5911424700 1 -- 172.21.15.47:0/2553668016 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f590c0b07f0 msgr2=0x7f590c157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5911424700 1 --2- 172.21.15.47:0/2553668016 >> [v2:172.21.15.202:3301/0,v1:172.21.15.202:6790/0] conn(0x7f590c0b07f0 0x7f590c157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5911424700 1 -- 172.21.15.47:0/2553668016 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f590c159310 msgr2=0x7f590c0c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T17:04:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5911424700 1 --2- 172.21.15.47:0/2553668016 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f590c159310 0x7f590c0c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T17:04:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5911424700 1 -- 172.21.15.47:0/2553668016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4009420 con 0x7f590c157950 2023-06-22T17:04:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f590bfff700 1 --2- 172.21.15.47:0/2553668016 >> [v2:172.21.15.202:3300/0,v1:172.21.15.202:6789/0] conn(0x7f590c159310 0x7f590c0c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T17:04:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5911424700 1 --2- 172.21.15.47:0/2553668016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f590c157950 0x7f590c0b3c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58f40093f0 tx=0x7f58f4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5908ff9700 1 -- 172.21.15.47:0/2553668016 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f401b450 con 0x7f590c157950 2023-06-22T17:04:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5908ff9700 1 -- 172.21.15.47:0/2553668016 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f4020070 con 0x7f590c157950 2023-06-22T17:04:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5911c25700 1 -- 172.21.15.47:0/2553668016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c16d100 con 0x7f590c157950 2023-06-22T17:04:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5908ff9700 1 -- 172.21.15.47:0/2553668016 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f401b820 con 0x7f590c157950 2023-06-22T17:04:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.184+0000 7f5911c25700 1 -- 172.21.15.47:0/2553668016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c159cb0 con 0x7f590c157950 2023-06-22T17:04:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.185+0000 7f5911c25700 1 -- 172.21.15.47:0/2553668016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c15dda0 con 0x7f590c157950 2023-06-22T17:04:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.185+0000 7f5908ff9700 1 -- 172.21.15.47:0/2553668016 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f58f4022030 con 0x7f590c157950 2023-06-22T17:04:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.186+0000 7f5908ff9700 1 --2- 172.21.15.47:0/2553668016 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f58ec066e90 0x7f58ec069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T17:04:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.186+0000 7f5908ff9700 1 -- 172.21.15.47:0/2553668016 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f58f4086710 con 0x7f590c157950 2023-06-22T17:04:15.192 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.186+0000 7f5910c23700 1 --2- 172.21.15.47:0/2553668016 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f58ec066e90 0x7f58ec069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T17:04:15.192 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.187+0000 7f5910c23700 1 --2- 172.21.15.47:0/2553668016 >> [v2:172.21.15.202:6832/87614,v1:172.21.15.202:6833/87614] conn(0x7f58ec066e90 0x7f58ec069340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5904007fd0 tx=0x7f5904009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T17:04:15.192 INFO:teuthology.orchestra.run.smithi047.stderr:2023-06-22T17:04:15.190+0000 7f5908ff9700 1 -- 172.21.15.47:0/2553668016 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f58f40538e0 con 0x7f590c157950 2023-06-22T17:04:15.436 INFO:teuthology.orchestra.run.smithi047.stderr:no valid command found; 1 closest matches: 2023-06-22T17:04:15.436 INFO:teuthology.orchestra.run.smithi047.stderr:fs set |refuse_client_session|max_xattr_size [--yes-i-really-mean-it] [--yes-i-really-really-mean-it] 2023-06-22T17:04:15.436 INFO:teuthology.orchestra.run.smithi047.stderr:Error EINVAL: invalid command 2023-06-22T17:04:15.439 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-06-22T17:04:15.440 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 450, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 717, in create self.set_allow_standby_replay(standby_replay) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 589, in set_allow_standby_replay self.set_var("allow_standby_replay", yes) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi047 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false' 2023-06-22T17:04:15.440 INFO:teuthology.misc:Shutting down mds daemons... 2023-06-22T17:04:15.441 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-22T17:04:15.441 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.449 INFO:tasks.ceph.mds.a:Stopped 2023-06-22T17:04:15.450 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-22T17:04:15.450 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.499 INFO:tasks.ceph.mds.c:Stopped 2023-06-22T17:04:15.500 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-22T17:04:15.500 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.510 INFO:tasks.ceph.mds.b:Stopped 2023-06-22T17:04:15.511 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-22T17:04:15.511 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.561 INFO:tasks.ceph.mds.d:Stopped 2023-06-22T17:04:15.562 INFO:teuthology.misc:Shutting down osd daemons... 2023-06-22T17:04:15.562 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-06-22T17:04:15.562 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.590 INFO:tasks.ceph.osd.0:Stopped 2023-06-22T17:04:15.591 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-06-22T17:04:15.591 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.658 INFO:tasks.ceph.osd.1:Stopped 2023-06-22T17:04:15.659 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-06-22T17:04:15.659 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.727 INFO:tasks.ceph.osd.2:Stopped 2023-06-22T17:04:15.727 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-06-22T17:04:15.728 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.796 INFO:tasks.ceph.osd.3:Stopped 2023-06-22T17:04:15.797 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-06-22T17:04:15.797 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.839 INFO:tasks.ceph.osd.4:Stopped 2023-06-22T17:04:15.840 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-06-22T17:04:15.840 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.871 INFO:tasks.ceph.osd.5:Stopped 2023-06-22T17:04:15.872 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-06-22T17:04:15.873 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:15.939 INFO:tasks.ceph.osd.6:Stopped 2023-06-22T17:04:15.939 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-06-22T17:04:15.940 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:16.007 INFO:tasks.ceph.osd.7:Stopped 2023-06-22T17:04:16.008 INFO:teuthology.misc:Shutting down mgr daemons... 2023-06-22T17:04:16.009 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-06-22T17:04:16.009 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:16.032 INFO:tasks.ceph.mgr.y:Stopped 2023-06-22T17:04:16.033 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-06-22T17:04:16.033 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:16.073 INFO:tasks.ceph.mgr.x:Stopped 2023-06-22T17:04:16.074 INFO:teuthology.misc:Shutting down mon daemons... 2023-06-22T17:04:16.074 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-06-22T17:04:16.074 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:16.085 INFO:tasks.ceph.mon.a:Stopped 2023-06-22T17:04:16.085 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-06-22T17:04:16.086 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:16.123 INFO:tasks.ceph.mon.b:Stopped 2023-06-22T17:04:16.124 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-06-22T17:04:16.125 INFO:teuthology.orchestra.run:waiting for 300 2023-06-22T17:04:16.175 INFO:tasks.ceph.mon.c:Stopped 2023-06-22T17:04:16.175 INFO:tasks.ceph:Checking cluster log for badness... 2023-06-22T17:04:16.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-06-22T17:04:16.207 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T17:04:16.207 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-06-22T17:04:16.562 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T17:04:16.563 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-06-22T17:04:16.806 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T17:04:16.807 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-06-22T17:04:16.888 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T17:04:16.888 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-06-22T17:04:16.968 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T17:04:16.969 DEBUG:teuthology.orchestra.run.smithi202:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-06-22T17:04:17.396 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T17:04:17.397 DEBUG:teuthology.orchestra.run.smithi202:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-06-22T17:04:17.485 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T17:04:17.485 DEBUG:teuthology.orchestra.run.smithi202:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-06-22T17:04:17.549 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T17:04:17.550 DEBUG:teuthology.orchestra.run.smithi202:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-06-22T17:04:17.608 INFO:tasks.ceph:Archiving mon data... 2023-06-22T17:04:17.612 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814/data/mon.a.tgz 2023-06-22T17:04:17.613 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2023-06-22T17:04:17.630 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.pNo5RsdIrm 2023-06-22T17:04:17.631 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.pNo5RsdIrm 2023-06-22T17:04:17.743 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.pNo5RsdIrm 2023-06-22T17:04:17.829 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.pNo5RsdIrm is 202KB 2023-06-22T17:04:17.852 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.pNo5RsdIrm 2023-06-22T17:04:17.867 DEBUG:teuthology.misc:Transferring archived files from smithi202:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814/data/mon.b.tgz 2023-06-22T17:04:17.868 DEBUG:teuthology.orchestra.run.smithi202:> mktemp 2023-06-22T17:04:17.884 INFO:teuthology.orchestra.run.smithi202.stdout:/tmp/tmp.YG7eGA0PsK 2023-06-22T17:04:17.885 DEBUG:teuthology.orchestra.run.smithi202:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.YG7eGA0PsK 2023-06-22T17:04:18.009 DEBUG:teuthology.orchestra.run.smithi202:> sudo chmod 0666 /tmp/tmp.YG7eGA0PsK 2023-06-22T17:04:18.091 DEBUG:teuthology.orchestra.remote:smithi202:/tmp/tmp.YG7eGA0PsK is 280KB 2023-06-22T17:04:18.182 DEBUG:teuthology.orchestra.run.smithi202:> rm -fr /tmp/tmp.YG7eGA0PsK 2023-06-22T17:04:18.196 DEBUG:teuthology.misc:Transferring archived files from smithi202:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814/data/mon.c.tgz 2023-06-22T17:04:18.197 DEBUG:teuthology.orchestra.run.smithi202:> mktemp 2023-06-22T17:04:18.250 INFO:teuthology.orchestra.run.smithi202.stdout:/tmp/tmp.orsa4sPHsN 2023-06-22T17:04:18.250 DEBUG:teuthology.orchestra.run.smithi202:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.orsa4sPHsN 2023-06-22T17:04:18.358 DEBUG:teuthology.orchestra.run.smithi202:> sudo chmod 0666 /tmp/tmp.orsa4sPHsN 2023-06-22T17:04:18.439 DEBUG:teuthology.orchestra.remote:smithi202:/tmp/tmp.orsa4sPHsN is 280KB 2023-06-22T17:04:18.509 DEBUG:teuthology.orchestra.run.smithi202:> rm -fr /tmp/tmp.orsa4sPHsN 2023-06-22T17:04:18.524 INFO:tasks.ceph:Cleaning ceph cluster... 2023-06-22T17:04:18.525 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-22T17:04:18.532 DEBUG:teuthology.orchestra.run.smithi202:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-22T17:04:18.587 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-06-22T17:04:18.587 DEBUG:teuthology.orchestra.run.smithi047:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-22T17:04:18.600 DEBUG:teuthology.orchestra.run.smithi202:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-22T17:04:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-22T17:04:18.652 INFO:teuthology.orchestra.run.smithi202.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-22T17:04:18.657 INFO:tasks.ceph:Archiving crash dumps... 2023-06-22T17:04:18.659 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814/remote/smithi047/crash 2023-06-22T17:04:18.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-22T17:04:18.699 DEBUG:teuthology.misc:Transferring archived files from smithi202:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814/remote/smithi202/crash 2023-06-22T17:04:18.701 DEBUG:teuthology.orchestra.run.smithi202:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-22T17:04:18.724 INFO:tasks.ceph:Shutting down logrotate 2023-06-22T17:04:18.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-22T17:04:18.745 DEBUG:teuthology.orchestra.run.smithi202:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-22T17:04:18.819 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-22T17:04:18.846 DEBUG:teuthology.orchestra.run.smithi202:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-22T17:04:18.870 INFO:tasks.ceph:Compressing logs... 2023-06-22T17:04:18.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-22T17:04:18.889 DEBUG:teuthology.orchestra.run.smithi202:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-22T17:04:19.184 INFO:tasks.ceph:Archiving logs... 2023-06-22T17:04:19.185 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814/remote/smithi047/log 2023-06-22T17:04:19.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-22T17:04:19.269 DEBUG:teuthology.misc:Transferring archived files from smithi202:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814/remote/smithi202/log 2023-06-22T17:04:19.270 DEBUG:teuthology.orchestra.run.smithi202:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-22T17:04:19.328 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 1893, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 450, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 717, in create self.set_allow_standby_replay(standby_replay) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 589, in set_allow_standby_replay self.set_var("allow_standby_replay", yes) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi047 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false' 2023-06-22T17:04:19.407 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=eed02eafee054b7cb3aa29fba3ee9748 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 1893, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 450, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 717, in create self.set_allow_standby_replay(standby_replay) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 589, in set_allow_standby_replay self.set_var("allow_standby_replay", yes) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi047 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false' 2023-06-22T17:04:19.413 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-06-22T17:04:19.426 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-06-22T17:04:19.455 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-06-22T17:04:19.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-22T17:04:19.487 DEBUG:teuthology.orchestra.run.smithi202:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-22T17:04:19.567 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-06-22T17:04:19.576 INFO:teuthology.task.clock:Checking final clock skew... 2023-06-22T17:04:19.576 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-22T17:04:19.579 DEBUG:teuthology.orchestra.run.smithi202:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-22T17:04:19.591 INFO:teuthology.orchestra.run.smithi047.stderr:bash: ntpq: command not found 2023-06-22T17:04:19.592 INFO:teuthology.orchestra.run.smithi202.stderr:bash: ntpq: command not found 2023-06-22T17:04:19.594 INFO:teuthology.orchestra.run.smithi047.stdout:210 Number of sources = 4 2023-06-22T17:04:19.594 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-22T17:04:19.595 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2023-06-22T17:04:19.595 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv01.front.sepia.ceph.com 2 6 177 46 +1609us[+1654us] +/- 91ms 2023-06-22T17:04:19.595 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv02.front.sepia.ceph.com 3 6 177 45 -2356us[-2312us] +/- 70ms 2023-06-22T17:04:19.595 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 47 +1246us[+1291us] +/- 73ms 2023-06-22T17:04:19.595 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:04:19.597 INFO:teuthology.orchestra.run.smithi202.stdout:210 Number of sources = 4 2023-06-22T17:04:19.597 INFO:teuthology.orchestra.run.smithi202.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-22T17:04:19.597 INFO:teuthology.orchestra.run.smithi202.stdout:=============================================================================== 2023-06-22T17:04:19.597 INFO:teuthology.orchestra.run.smithi202.stdout:^+ hv01.front.sepia.ceph.com 2 6 177 47 +1614us[+1723us] +/- 91ms 2023-06-22T17:04:19.598 INFO:teuthology.orchestra.run.smithi202.stdout:^* hv02.front.sepia.ceph.com 3 6 177 45 -2326us[-2216us] +/- 70ms 2023-06-22T17:04:19.598 INFO:teuthology.orchestra.run.smithi202.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 45 +1270us[+1270us] +/- 73ms 2023-06-22T17:04:19.598 INFO:teuthology.orchestra.run.smithi202.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:04:19.599 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-06-22T17:04:19.607 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-06-22T17:04:19.608 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-06-22T17:04:19.639 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-22T17:04:19.729 DEBUG:teuthology.orchestra.run.smithi202:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-22T17:04:19.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-22T17:04:19.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T17:04:19.844 DEBUG:teuthology.orchestra.run.smithi202:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-22T17:04:19.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T17:04:19.874 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-06-22T17:04:19.882 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-06-22T17:04:19.911 INFO:teuthology.task.internal:Duration was 718.871732 seconds 2023-06-22T17:04:19.911 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-06-22T17:04:19.940 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-06-22T17:04:19.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-22T17:04:19.944 DEBUG:teuthology.orchestra.run.smithi202:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-22T17:04:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-22T17:04:19.987 INFO:teuthology.orchestra.run.smithi202.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-22T17:04:20.304 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-06-22T17:04:20.304 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2023-06-22T17:04:20.305 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-22T17:04:20.365 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi202.front.sepia.ceph.com 2023-06-22T17:04:20.365 DEBUG:teuthology.orchestra.run.smithi202:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-22T17:04:20.384 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-06-22T17:04:20.385 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-22T17:04:20.407 DEBUG:teuthology.orchestra.run.smithi202:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-22T17:04:20.463 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-06-22T17:04:20.464 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-22T17:04:20.487 DEBUG:teuthology.orchestra.run.smithi202:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-22T17:04:20.572 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-06-22T17:04:20.586 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-06-22T17:04:20.587 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-22T17:04:20.612 DEBUG:teuthology.orchestra.run.smithi202:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-22T17:04:20.636 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-06-22T17:04:20.646 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-22T17:04:20.655 DEBUG:teuthology.orchestra.run.smithi202:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-22T17:04:20.677 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2023-06-22T17:04:20.699 INFO:teuthology.orchestra.run.smithi202.stdout:kernel.core_pattern = core 2023-06-22T17:04:20.713 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-22T17:04:20.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T17:04:20.746 DEBUG:teuthology.orchestra.run.smithi202:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-22T17:04:20.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T17:04:20.767 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-06-22T17:04:20.777 INFO:teuthology.task.internal:Transferring archived files... 2023-06-22T17:04:20.778 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814/remote/smithi047 2023-06-22T17:04:20.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-22T17:04:20.820 DEBUG:teuthology.misc:Transferring archived files from smithi202:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814/remote/smithi202 2023-06-22T17:04:20.820 DEBUG:teuthology.orchestra.run.smithi202:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-22T17:04:20.856 INFO:teuthology.task.internal:Removing archive directory... 2023-06-22T17:04:20.856 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-22T17:04:20.858 DEBUG:teuthology.orchestra.run.smithi202:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-22T17:04:20.905 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-06-22T17:04:20.917 INFO:teuthology.task.internal:Not uploading archives. 2023-06-22T17:04:20.917 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-06-22T17:04:20.949 INFO:teuthology.task.internal:Tidying up after the test... 2023-06-22T17:04:20.950 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-22T17:04:20.952 DEBUG:teuthology.orchestra.run.smithi202:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-22T17:04:20.964 INFO:teuthology.orchestra.run.smithi047.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 22 17:04 /home/ubuntu/cephtest 2023-06-22T17:04:20.965 INFO:teuthology.orchestra.run.smithi202.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 22 17:04 /home/ubuntu/cephtest 2023-06-22T17:04:20.967 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-06-22T17:04:20.981 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-06-22T17:04:21.057 INFO:teuthology.nuke:Checking targets against current locks 2023-06-22T17:04:21.088 DEBUG:teuthology.nuke:shortname: smithi047 2023-06-22T17:04:21.089 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T17:04:21.115 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 16:45:04.550447', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlSCLbZxCowylv4ejcGPoLZwSzztkWsoXph8jwxtsLJ3VXlph3F9wrR1ghn5qjjUtVXhuKxEgDbyX2uINaPFag='} 2023-06-22T17:04:21.120 DEBUG:teuthology.nuke:shortname: smithi202 2023-06-22T17:04:21.120 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T17:04:21.142 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi202.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311814', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 16:45:04.552540', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDypvRW1ScAmzbCqMdu2b44iXhoAYmmsMtSCXCqMi+j9mOvi4XOcXfKNu/BGvrd49WtXo1PRU4OHW4zMvGMex9eKutRgJWxD9Y3HnFOXkfIaKeFe6odwFQ1RiPLLCSWoeZ5GeIqpqEs9xMEusoo4ly25clw6Ex+7chqlCXWcqt3aVasVUB+46NzjXHQtiPfZn9euC07pub6yE5fowjee3DrA/f8wRRhcXr6EZPuupJkEwQlgwz+dtEA9TG6x/WbPTqyYZ6O4B0Qv9/7IavdhpBLFwH/RbKwnMHDc4drAWjZf+WX65YOMNDvCejLaXT2eyStMp8JisTVu8vXlvUZ7fQjJ29WD3pvzjxHKh5mQdQbYV0m3L2EEBZvLDEidx0YJmlXkZcanoRDbSEeEMHuUP/5nc2OU7OmdEleiydSK1jfVkVyYAbfz7RNpajUaBehbvyxnB5d1Oy1E41DoIEnjIks5Ee4tj6Im2iqIA4bhjyMi8rCjelVj5/8xoJb2AteqYM='} 2023-06-22T17:04:21.172 INFO:teuthology.orchestra.console:Power off smithi047 2023-06-22T17:04:21.173 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T17:04:21.198 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:04:21.217 INFO:teuthology.orchestra.console:Power off smithi202 2023-06-22T17:04:21.218 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T17:04:21.243 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:04:25.230 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:04:25.266 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:04:29.348 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:04:29.379 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:04:29.561 INFO:teuthology.orchestra.console:Power off for smithi047 completed 2023-06-22T17:04:33.492 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:04:33.704 INFO:teuthology.orchestra.console:Power off for smithi202 completed 2023-06-22T17:04:33.815 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} duration: 718.8717315196991 failure_reason: 'Command failed on smithi047 with status 22: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=eed02eafee054b7cb3aa29fba3ee9748 status: fail success: false 2023-06-22T17:04:33.816 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T17:04:34.144 INFO:teuthology.run:FAIL