2022-09-20T19:56:08.258 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-20T19:56:08.260 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2022-09-20_19:41:25-fs:functional-wip-pdonnell-testing-20220920.151129-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2022-09-20_19:41:25-fs:functional-wip-pdonnell-testing-20220920.151129-distro-default-smithi/7039471 --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} -- /home/teuthworker/archive/pdonnell-2022-09-20_19:41:25-fs:functional-wip-pdonnell-testing-20220920.151129-distro-default-smithi/7039471/orig.config.yaml 2022-09-20T19:56:08.308 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-09-20T19:56:08.379 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2022-09-20_19:41:25-fs:functional-wip-pdonnell-testing-20220920.151129-distro-default-smithi/7039471 branch: wip-pdonnell-testing-20220920.151129 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} email: pdonnell@redhat.com first_in_suite: false job_id: '7039471' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2022-09-20_19:41:25-fs:functional-wip-pdonnell-testing-20220920.151129-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20220920.151129 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - MDS_FAILED - MDS_DAMAGE sha1: ada1013c357729fc3bf91fdc0bbe1d411f5af166 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ada1013c357729fc3bf91fdc0bbe1d411f5af166 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20220920.151129 sha1: 7874f3907d0b2e2027e668918a6854269060504f owner: scheduled_pdonnell@teuthology priority: 50 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: ada1013c357729fc3bf91fdc0bbe1d411f5af166 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-pdonnell-testing-20220920.151129 suite_path: /home/teuthworker/src/github.com_batrick_ceph_7874f3907d0b2e2027e668918a6854269060504f/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: 7874f3907d0b2e2027e668918a6854269060504f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi036.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb7XvCj6ppza8aYF22opmyta1A2KNqYjUn8APLjXM6u4BOCCwjrDMJ2YRUq0pwsj6GqrFcpKh33X1Wycqe+EsHk/wZGitUexLIrdvcgu4K7+VkTRkssYrqIaEwT1GSRyLF667txFJpg/UAN6FBck97qimlM5eEAdoAtRClciIRQZIUlt8PDSp3ZzU+/Y6jbwxtREp0WG/DWUOroPY25j3vHwYnjBlHyCjR84aVxf9EqjQMCYPOPwbblf6iAfsSvVKPTpbS0Qv17mE9MCZI0IIQrTgrJdCDZtk+BBz0Rn0K2jX0C8CdCzWH8eVlv8kAL+ZLAMEcH9+jxwGU20tG71/r6HPCYCX2CbZmeUnkRwl5hPKd1RJV//ae/k6tF8weeR5f1FcAgXAkzqvnhLjutDHS+nhqN3ggVfHxdh7Mt9Zyo+iJBz1ijD9ncHrPlsdWrYnHqFJ+e4hl3NnWNU6vrFNXcp3ks/oPF6RoxvCBma10LKE6thbcld81LHlbZVrNFnk= smithi115.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDS0vHkhkc7dtzk8QNT7mI8egs4xNS98blr0rI/LG0d9zDyaTqxeVFnQcKSplSy4wcAkBfu87Px4ePWEB7C/dB4bpKS+gyunsdbN4lfmVs1ce84iUMrt/LgZvIdFqmG6fXCYaCdqAZCK1CLGjvydAGz1jgwadrAprI3zPNNAIOuFh4bQsvVj+tbX9hJ5P8zfb1mtGzwZQc7iwu0mEwBmmbQlEGOFI258soNPFGT9QgVoZKgHUvms+4PCEOW1k6sVCPmfsZI5Jc+GOnf8ElIRlFImgY1yPHehvKeSu9G37XVWlqc/2SpGW/7evNSqK4NLzO9L1QaO/2fLMZL/PEyfSdLJakCIT4QcdnJHYufWx4VuAeyhWOAzVcCMdSes3dcmDYhAaZB3Tg9zreDT7Pi/y9SupY8+UrGAcVnsQY9QxhQHE23jbAUUU01E3078MoDqS5WAsi3Vtrc0Hom315zY+vO30CkFGrTQo0OeBrc4mHZ7V3FIUag7NlPF6UN/VLY2Gk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_recovery_pool teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b58f2c18636eb10faa77ed3614abd00cb85dfc2c timestamp: 2022-09-20_19:41:25 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.14413 2022-09-20T19:56:08.380 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_7874f3907d0b2e2027e668918a6854269060504f/qa; will attempt to use it 2022-09-20T19:56:08.381 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_7874f3907d0b2e2027e668918a6854269060504f/qa/tasks 2022-09-20T19:56:08.381 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-09-20T19:56:08.383 INFO:teuthology.task.internal:Checking packages... 2022-09-20T19:56:08.410 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'ada1013c357729fc3bf91fdc0bbe1d411f5af166' 2022-09-20T19:56:08.410 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-09-20T19:56:08.411 INFO:teuthology.packaging:ref: None 2022-09-20T19:56:08.411 INFO:teuthology.packaging:tag: None 2022-09-20T19:56:08.412 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20220920.151129 2022-09-20T19:56:08.412 INFO:teuthology.packaging:sha1: ada1013c357729fc3bf91fdc0bbe1d411f5af166 2022-09-20T19:56:08.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20220920.151129 2022-09-20T19:56:08.716 INFO:teuthology.task.internal:Found packages for ceph version 17.0.0-15134.gada1013c 2022-09-20T19:56:08.716 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-09-20T19:56:08.723 INFO:teuthology.task.internal:no buildpackages task found 2022-09-20T19:56:08.724 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-09-20T19:56:08.730 INFO:teuthology.task.internal:Saving configuration 2022-09-20T19:56:08.747 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-09-20T19:56:08.759 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-20T19:56:08.784 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi036.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2022-09-20_19:41:25-fs:functional-wip-pdonnell-testing-20220920.151129-distro-default-smithi/7039471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-09-20 19:51:08.757343', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPMRT3qM9Ldvno7nWgxwyaifvVKYeA/svoLgdd//5PSkqDQHMnd90pX8MGndbZmUmoJjC+2a6Zw84O8imC8o9Ks='} 2022-09-20T19:56:08.806 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2022-09-20_19:41:25-fs:functional-wip-pdonnell-testing-20220920.151129-distro-default-smithi/7039471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-09-20 19:51:08.758918', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOWPr5/y7MZehb6u2UKZ6IcWA+CmgYzqTnQoomjg2IvYaMWuAih6/KJ4+wR+e6cnSS3JB6+a1rS4sGcM1N7M9Fk='} 2022-09-20T19:56:08.806 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-09-20T19:56:08.813 INFO:teuthology.task.internal:roles: ubuntu@smithi036.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-09-20T19:56:08.813 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-09-20T19:56:08.813 INFO:teuthology.run_tasks:Running task console_log... 2022-09-20T19:56:08.904 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd0ecff8ea0>, signals=[15]) 2022-09-20T19:56:08.905 INFO:teuthology.run_tasks:Running task internal.connect... 2022-09-20T19:56:08.912 INFO:teuthology.task.internal:Opening connections... 2022-09-20T19:56:08.912 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi036.front.sepia.ceph.com 2022-09-20T19:56:08.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-09-20T19:56:08.996 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2022-09-20T19:56:08.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-09-20T19:56:09.069 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-09-20T19:56:09.077 DEBUG:teuthology.orchestra.run.smithi036:> uname -m 2022-09-20T19:56:09.098 INFO:teuthology.orchestra.run.smithi036.stdout:x86_64 2022-09-20T19:56:09.098 DEBUG:teuthology.orchestra.run.smithi036:> cat /etc/os-release 2022-09-20T19:56:09.158 INFO:teuthology.orchestra.run.smithi036.stdout:NAME="Red Hat Enterprise Linux" 2022-09-20T19:56:09.158 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION="8.6 (Ootpa)" 2022-09-20T19:56:09.159 INFO:teuthology.orchestra.run.smithi036.stdout:ID="rhel" 2022-09-20T19:56:09.159 INFO:teuthology.orchestra.run.smithi036.stdout:ID_LIKE="fedora" 2022-09-20T19:56:09.159 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION_ID="8.6" 2022-09-20T19:56:09.159 INFO:teuthology.orchestra.run.smithi036.stdout:PLATFORM_ID="platform:el8" 2022-09-20T19:56:09.159 INFO:teuthology.orchestra.run.smithi036.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-20T19:56:09.160 INFO:teuthology.orchestra.run.smithi036.stdout:ANSI_COLOR="0;31" 2022-09-20T19:56:09.160 INFO:teuthology.orchestra.run.smithi036.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-20T19:56:09.160 INFO:teuthology.orchestra.run.smithi036.stdout:HOME_URL="https://www.redhat.com/" 2022-09-20T19:56:09.160 INFO:teuthology.orchestra.run.smithi036.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-20T19:56:09.160 INFO:teuthology.orchestra.run.smithi036.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-20T19:56:09.161 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T19:56:09.161 INFO:teuthology.orchestra.run.smithi036.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-20T19:56:09.161 INFO:teuthology.orchestra.run.smithi036.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-20T19:56:09.161 INFO:teuthology.orchestra.run.smithi036.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-20T19:56:09.162 INFO:teuthology.orchestra.run.smithi036.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-20T19:56:09.162 INFO:teuthology.lock.ops:Updating smithi036.front.sepia.ceph.com on lock server 2022-09-20T19:56:09.182 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2022-09-20T19:56:09.203 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2022-09-20T19:56:09.203 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2022-09-20T19:56:09.261 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="Red Hat Enterprise Linux" 2022-09-20T19:56:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="8.6 (Ootpa)" 2022-09-20T19:56:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:ID="rhel" 2022-09-20T19:56:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="fedora" 2022-09-20T19:56:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="8.6" 2022-09-20T19:56:09.263 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el8" 2022-09-20T19:56:09.263 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-09-20T19:56:09.263 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2022-09-20T19:56:09.263 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-09-20T19:56:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.redhat.com/" 2022-09-20T19:56:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-09-20T19:56:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-20T19:56:09.264 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T19:56:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-20T19:56:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-09-20T19:56:09.265 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-09-20T19:56:09.265 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-09-20T19:56:09.265 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2022-09-20T19:56:09.283 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-09-20T19:56:09.293 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-09-20T19:56:09.299 INFO:teuthology.task.internal:Checking for old test directory... 2022-09-20T19:56:09.299 DEBUG:teuthology.orchestra.run.smithi036:> test '!' -e /home/ubuntu/cephtest 2022-09-20T19:56:09.302 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2022-09-20T19:56:09.320 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-09-20T19:56:09.326 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-09-20T19:56:09.327 DEBUG:teuthology.orchestra.run.smithi036:> test -z $(ls -A /var/lib/ceph) 2022-09-20T19:56:09.361 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2022-09-20T19:56:09.390 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-09-20T19:56:09.447 INFO:teuthology.run_tasks:Running task kernel... 2022-09-20T19:56:09.461 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2022-09-20T19:56:09.462 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2022-09-20T19:56:09.462 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2022-09-20T19:56:09.462 DEBUG:teuthology.orchestra.run.smithi036:> test -f /run/.containerenv -o -f /.dockerenv 2022-09-20T19:56:09.463 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2022-09-20T19:56:09.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-20T19:56:09.481 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2022-09-20T19:56:09.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-20T19:56:09.484 DEBUG:teuthology.orchestra.run.smithi036:> uname -r 2022-09-20T19:56:09.539 INFO:teuthology.orchestra.run.smithi115.stdout:4.18.0-372.9.1.el8.x86_64 2022-09-20T19:56:09.540 INFO:teuthology.task.kernel:Running kernel on smithi115: 4.18.0-372.9.1.el8.x86_64 2022-09-20T19:56:09.540 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2022-09-20T19:56:09.543 INFO:teuthology.orchestra.run.smithi036.stdout:4.18.0-372.9.1.el8.x86_64 2022-09-20T19:56:09.544 INFO:teuthology.task.kernel:Running kernel on smithi036: 4.18.0-372.9.1.el8.x86_64 2022-09-20T19:56:09.544 DEBUG:teuthology.orchestra.run.smithi036:> sudo yum install -y kernel 2022-09-20T19:56:10.550 INFO:teuthology.orchestra.run.smithi036.stdout:Updating Subscription Management repositories. 2022-09-20T19:56:10.551 INFO:teuthology.orchestra.run.smithi036.stdout:Unable to read consumer identity 2022-09-20T19:56:10.551 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2022-09-20T19:56:10.552 INFO:teuthology.orchestra.run.smithi115.stdout:Unable to read consumer identity 2022-09-20T19:56:10.611 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T19:56:10.612 INFO:teuthology.orchestra.run.smithi115.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-09-20T19:56:10.612 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T19:56:10.616 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T19:56:10.617 INFO:teuthology.orchestra.run.smithi036.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-09-20T19:56:10.617 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T19:56:11.919 INFO:teuthology.orchestra.run.smithi115.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2022-09-20T19:56:11.989 INFO:teuthology.orchestra.run.smithi036.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2022-09-20T19:56:14.531 INFO:teuthology.orchestra.run.smithi036.stdout:Extra Packages for Enterprise Linux 5.1 MB/s | 13 MB 00:02 2022-09-20T19:56:16.844 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 2.6 MB/s | 13 MB 00:04 2022-09-20T19:56:17.581 INFO:teuthology.orchestra.run.smithi036.stdout:lab-extras 393 kB/s | 24 kB 00:00 2022-09-20T19:56:18.234 INFO:teuthology.orchestra.run.smithi036.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-09-20T19:56:18.287 INFO:teuthology.orchestra.run.smithi036.stdout:Dependencies resolved. 2022-09-20T19:56:18.288 INFO:teuthology.orchestra.run.smithi036.stdout:Nothing to do. 2022-09-20T19:56:18.289 INFO:teuthology.orchestra.run.smithi036.stdout:Complete! 2022-09-20T19:56:18.420 DEBUG:teuthology.orchestra.run.smithi036:> echo no | sudo yum reinstall kernel || true 2022-09-20T19:56:18.770 INFO:teuthology.orchestra.run.smithi036.stdout:Updating Subscription Management repositories. 2022-09-20T19:56:18.771 INFO:teuthology.orchestra.run.smithi036.stdout:Unable to read consumer identity 2022-09-20T19:56:18.813 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T19:56:18.814 INFO:teuthology.orchestra.run.smithi036.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-09-20T19:56:18.814 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T19:56:19.058 INFO:teuthology.orchestra.run.smithi036.stdout:Last metadata expiration check: 0:00:02 ago on Tue 20 Sep 2022 07:56:17 PM UTC. 2022-09-20T19:56:19.109 INFO:teuthology.orchestra.run.smithi036.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-09-20T19:56:19.138 INFO:teuthology.orchestra.run.smithi036.stderr:Error: No packages marked for reinstall. 2022-09-20T19:56:19.186 DEBUG:teuthology.orchestra.run.smithi036:> sudo yum reinstall -y kernel || true 2022-09-20T19:56:19.536 INFO:teuthology.orchestra.run.smithi036.stdout:Updating Subscription Management repositories. 2022-09-20T19:56:19.536 INFO:teuthology.orchestra.run.smithi036.stdout:Unable to read consumer identity 2022-09-20T19:56:19.576 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T19:56:19.577 INFO:teuthology.orchestra.run.smithi036.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-09-20T19:56:19.577 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T19:56:19.661 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 323 kB/s | 24 kB 00:00 2022-09-20T19:56:19.815 INFO:teuthology.orchestra.run.smithi036.stdout:Last metadata expiration check: 0:00:02 ago on Tue 20 Sep 2022 07:56:17 PM UTC. 2022-09-20T19:56:19.858 INFO:teuthology.orchestra.run.smithi036.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-09-20T19:56:19.880 INFO:teuthology.orchestra.run.smithi036.stderr:Error: No packages marked for reinstall. 2022-09-20T19:56:19.933 DEBUG:teuthology.orchestra.run.smithi036:> rpm -q kernel --last | head -n 1 2022-09-20T19:56:20.015 INFO:teuthology.orchestra.run.smithi036.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-09-20T19:56:20.018 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-09-20T19:56:20.018 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-09-20T19:56:20.019 DEBUG:teuthology.parallel:result is None 2022-09-20T19:56:20.322 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-09-20T19:56:20.371 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2022-09-20T19:56:20.372 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2022-09-20T19:56:20.372 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2022-09-20T19:56:20.503 DEBUG:teuthology.orchestra.run.smithi115:> echo no | sudo yum reinstall kernel || true 2022-09-20T19:56:20.835 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2022-09-20T19:56:20.835 INFO:teuthology.orchestra.run.smithi115.stdout:Unable to read consumer identity 2022-09-20T19:56:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T19:56:20.875 INFO:teuthology.orchestra.run.smithi115.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-09-20T19:56:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T19:56:21.103 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:02 ago on Tue 20 Sep 2022 07:56:19 PM UTC. 2022-09-20T19:56:21.154 INFO:teuthology.orchestra.run.smithi115.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-09-20T19:56:21.191 INFO:teuthology.orchestra.run.smithi115.stderr:Error: No packages marked for reinstall. 2022-09-20T19:56:21.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum reinstall -y kernel || true 2022-09-20T19:56:21.566 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2022-09-20T19:56:21.566 INFO:teuthology.orchestra.run.smithi115.stdout:Unable to read consumer identity 2022-09-20T19:56:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T19:56:21.605 INFO:teuthology.orchestra.run.smithi115.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-09-20T19:56:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T19:56:21.829 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:02 ago on Tue 20 Sep 2022 07:56:19 PM UTC. 2022-09-20T19:56:21.868 INFO:teuthology.orchestra.run.smithi115.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-09-20T19:56:21.904 INFO:teuthology.orchestra.run.smithi115.stderr:Error: No packages marked for reinstall. 2022-09-20T19:56:21.954 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel --last | head -n 1 2022-09-20T19:56:22.033 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-09-20T19:56:22.035 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-09-20T19:56:22.035 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-09-20T19:56:22.036 DEBUG:teuthology.parallel:result is None 2022-09-20T19:56:22.036 INFO:teuthology.run_tasks:Running task internal.base... 2022-09-20T19:56:22.044 INFO:teuthology.task.internal:Creating test directory... 2022-09-20T19:56:22.045 DEBUG:teuthology.orchestra.run.smithi036:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-09-20T19:56:22.048 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-09-20T19:56:22.066 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-09-20T19:56:22.073 INFO:teuthology.run_tasks:Running task internal.archive... 2022-09-20T19:56:22.084 INFO:teuthology.task.internal:Creating archive directory... 2022-09-20T19:56:22.084 DEBUG:teuthology.orchestra.run.smithi036:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-09-20T19:56:22.109 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-09-20T19:56:22.144 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-09-20T19:56:22.151 INFO:teuthology.task.internal:Enabling coredump saving... 2022-09-20T19:56:22.152 DEBUG:teuthology.orchestra.run.smithi036:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-09-20T19:56:22.181 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-09-20T19:56:22.213 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-09-20T19:56:22.220 INFO:teuthology.orchestra.run.smithi036.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-09-20T19:56:22.228 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-09-20T19:56:22.235 INFO:teuthology.orchestra.run.smithi036.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-09-20T19:56:22.237 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-09-20T19:56:22.245 INFO:teuthology.task.internal:Configuring sudo... 2022-09-20T19:56:22.245 DEBUG:teuthology.orchestra.run.smithi036:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-09-20T19:56:22.280 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-09-20T19:56:22.309 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-09-20T19:56:22.318 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-09-20T19:56:22.318 DEBUG:teuthology.orchestra.run.smithi036:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-09-20T19:56:22.352 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-09-20T19:56:22.368 DEBUG:teuthology.orchestra.run.smithi036:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-09-20T19:56:22.454 DEBUG:teuthology.orchestra.run.smithi036:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-09-20T19:56:22.509 DEBUG:teuthology.orchestra.run.smithi036:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-09-20T19:56:22.593 DEBUG:teuthology.orchestra.run.smithi036:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-09-20T19:56:22.638 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T19:56:22.639 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-09-20T19:56:22.706 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-09-20T19:56:22.750 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-09-20T19:56:22.797 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-09-20T19:56:22.880 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-09-20T19:56:22.923 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T19:56:22.924 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-09-20T19:56:22.989 DEBUG:teuthology.orchestra.run.smithi036:> sudo service rsyslog restart 2022-09-20T19:56:22.992 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2022-09-20T19:56:23.046 INFO:teuthology.orchestra.run.smithi036.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-09-20T19:56:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-09-20T19:56:23.436 INFO:teuthology.run_tasks:Running task internal.timer... 2022-09-20T19:56:23.450 INFO:teuthology.task.internal:Starting timer... 2022-09-20T19:56:23.450 INFO:teuthology.run_tasks:Running task pcp... 2022-09-20T19:56:23.464 INFO:teuthology.run_tasks:Running task selinux... 2022-09-20T19:56:23.472 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2022-09-20T19:56:23.473 DEBUG:teuthology.orchestra.run.smithi036:> sudo service auditd rotate 2022-09-20T19:56:23.525 INFO:teuthology.orchestra.run.smithi036.stdout:Rotating logs: 2022-09-20T19:56:23.527 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2022-09-20T19:56:23.601 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2022-09-20T19:56:23.602 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-09-20T19:56:23.603 DEBUG:teuthology.orchestra.run.smithi036:> /usr/sbin/getenforce 2022-09-20T19:56:23.629 INFO:teuthology.orchestra.run.smithi036.stdout:Permissive 2022-09-20T19:56:23.629 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2022-09-20T19:56:23.652 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2022-09-20T19:56:23.653 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi036.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive'} 2022-09-20T19:56:23.653 DEBUG:teuthology.orchestra.run.smithi036:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-09-20T19:56:23.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-20T19:56:23.698 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-09-20T19:56:23.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-20T19:56:23.725 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-09-20T19:56:23.725 DEBUG:teuthology.orchestra.run.smithi036:> sudo /usr/sbin/setenforce permissive 2022-09-20T19:56:23.765 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2022-09-20T19:56:23.794 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-09-20T19:56:23.803 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2022-09-20T19:56:23.811 INFO:teuthology.repo_utils:Fetching main from origin 2022-09-20T19:56:23.850 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main to origin/main 2022-09-20T19:56:23.871 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-09-20T19:56:23.873 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi036.front.sepia.ceph.com,smithi115.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main/cephlab.yml 2022-09-20T20:04:02.586 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi036.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com')] 2022-09-20T20:04:02.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi036.front.sepia.ceph.com' 2022-09-20T20:04:02.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-09-20T20:04:02.677 DEBUG:teuthology.orchestra.run.smithi036:> true 2022-09-20T20:04:02.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2022-09-20T20:04:02.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-09-20T20:04:02.822 DEBUG:teuthology.orchestra.run.smithi115:> true 2022-09-20T20:04:02.893 INFO:teuthology.run_tasks:Running task clock... 2022-09-20T20:04:02.906 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-09-20T20:04:02.906 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-09-20T20:04:02.909 DEBUG:teuthology.orchestra.run.smithi036:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-09-20T20:04:02.912 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-09-20T20:04:02.912 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-09-20T20:04:02.944 INFO:teuthology.orchestra.run.smithi036.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-09-20T20:04:02.960 INFO:teuthology.orchestra.run.smithi036.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-09-20T20:04:02.966 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-09-20T20:04:02.981 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-09-20T20:04:03.034 INFO:teuthology.orchestra.run.smithi036.stderr:sudo: ntpd: command not found 2022-09-20T20:04:03.046 INFO:teuthology.orchestra.run.smithi036.stdout:506 Cannot talk to daemon 2022-09-20T20:04:03.049 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2022-09-20T20:04:03.061 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2022-09-20T20:04:03.062 INFO:teuthology.orchestra.run.smithi036.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-09-20T20:04:03.076 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-09-20T20:04:03.077 INFO:teuthology.orchestra.run.smithi036.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-09-20T20:04:03.090 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-09-20T20:04:03.151 INFO:teuthology.orchestra.run.smithi036.stderr:bash: ntpq: command not found 2022-09-20T20:04:03.154 INFO:teuthology.orchestra.run.smithi036.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-09-20T20:04:03.155 INFO:teuthology.orchestra.run.smithi036.stdout:=============================================================================== 2022-09-20T20:04:03.155 INFO:teuthology.orchestra.run.smithi036.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-20T20:04:03.155 INFO:teuthology.orchestra.run.smithi036.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-20T20:04:03.155 INFO:teuthology.orchestra.run.smithi036.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-20T20:04:03.156 INFO:teuthology.orchestra.run.smithi036.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-20T20:04:03.164 INFO:teuthology.orchestra.run.smithi115.stderr:bash: ntpq: command not found 2022-09-20T20:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-09-20T20:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2022-09-20T20:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-20T20:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-20T20:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-20T20:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-20T20:04:03.170 INFO:teuthology.run_tasks:Running task install... 2022-09-20T20:04:03.179 DEBUG:teuthology.task.install:project ceph 2022-09-20T20:04:03.179 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ada1013c357729fc3bf91fdc0bbe1d411f5af166'}} 2022-09-20T20:04:03.180 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ada1013c357729fc3bf91fdc0bbe1d411f5af166'} 2022-09-20T20:04:03.180 INFO:teuthology.task.install:Using flavor: default 2022-09-20T20:04:03.188 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-09-20T20:04:03.188 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2022-09-20T20:04:03.189 DEBUG:teuthology.orchestra.run.smithi036:> sudo lsb_release -is 2022-09-20T20:04:03.238 INFO:teuthology.orchestra.run.smithi036.stdout:RedHatEnterprise 2022-09-20T20:04:03.240 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-09-20T20:04:03.241 DEBUG:teuthology.orchestra.run.smithi115:> sudo lsb_release -is 2022-09-20T20:04:03.241 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ada1013c357729fc3bf91fdc0bbe1d411f5af166', 'tag': None, 'wait_for_package': False} 2022-09-20T20:04:03.242 DEBUG:teuthology.orchestra.run.smithi036:> sudo lsb_release -is 2022-09-20T20:04:03.285 INFO:teuthology.orchestra.run.smithi115.stdout:RedHatEnterprise 2022-09-20T20:04:03.286 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-09-20T20:04:03.286 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ada1013c357729fc3bf91fdc0bbe1d411f5af166', 'tag': None, 'wait_for_package': False} 2022-09-20T20:04:03.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo lsb_release -is 2022-09-20T20:04:03.322 INFO:teuthology.orchestra.run.smithi036.stdout:RedHatEnterprise 2022-09-20T20:04:03.323 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-09-20T20:04:03.324 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ada1013c357729fc3bf91fdc0bbe1d411f5af166 2022-09-20T20:04:03.369 INFO:teuthology.orchestra.run.smithi115.stdout:RedHatEnterprise 2022-09-20T20:04:03.370 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-09-20T20:04:03.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ada1013c357729fc3bf91fdc0bbe1d411f5af166 2022-09-20T20:04:03.493 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20220920.151129/ada1013c357729fc3bf91fdc0bbe1d411f5af166/centos/8/flavors/default/ 2022-09-20T20:04:03.494 INFO:teuthology.task.install.rpm:Package version is 17.0.0-15134.gada1013c 2022-09-20T20:04:03.596 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20220920.151129/ada1013c357729fc3bf91fdc0bbe1d411f5af166/centos/8/flavors/default/ 2022-09-20T20:04:03.597 INFO:teuthology.task.install.rpm:Package version is 17.0.0-15134.gada1013c 2022-09-20T20:04:03.725 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20220920.151129/ada1013c357729fc3bf91fdc0bbe1d411f5af166/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20220920.151129/ada1013c357729fc3bf91fdc0bbe1d411f5af166/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20220920.151129/ada1013c357729fc3bf91fdc0bbe1d411f5af166/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-09-20T20:04:03.726 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:04:03.726 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-09-20T20:04:03.728 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20220920.151129/ada1013c357729fc3bf91fdc0bbe1d411f5af166/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20220920.151129/ada1013c357729fc3bf91fdc0bbe1d411f5af166/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20220920.151129/ada1013c357729fc3bf91fdc0bbe1d411f5af166/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-09-20T20:04:03.729 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:04:03.729 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-09-20T20:04:03.756 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-09-20T20:04:03.757 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ada1013c357729fc3bf91fdc0bbe1d411f5af166/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-09-20T20:04:03.758 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-09-20T20:04:03.759 DEBUG:teuthology.orchestra.run.smithi036:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ada1013c357729fc3bf91fdc0bbe1d411f5af166/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-09-20T20:04:03.824 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-09-20T20:04:03.827 DEBUG:teuthology.orchestra.run.smithi036:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-09-20T20:04:03.901 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-09-20T20:04:03.906 DEBUG:teuthology.orchestra.run.smithi036:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-09-20T20:04:03.967 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2022-09-20T20:04:03.969 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2022-09-20T20:04:03.973 INFO:teuthology.orchestra.run.smithi036.stdout:check_obsoletes = 1 2022-09-20T20:04:03.975 DEBUG:teuthology.orchestra.run.smithi036:> sudo yum clean all 2022-09-20T20:04:04.339 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2022-09-20T20:04:04.367 INFO:teuthology.orchestra.run.smithi036.stdout:Updating Subscription Management repositories. 2022-09-20T20:04:04.534 INFO:teuthology.orchestra.run.smithi115.stdout:52 files removed 2022-09-20T20:04:04.563 INFO:teuthology.orchestra.run.smithi036.stdout:52 files removed 2022-09-20T20:04:04.581 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-09-20T20:04:04.611 DEBUG:teuthology.orchestra.run.smithi036:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-09-20T20:04:04.911 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2022-09-20T20:04:04.960 INFO:teuthology.orchestra.run.smithi036.stdout:Updating Subscription Management repositories. 2022-09-20T20:04:05.522 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 261 kB/s | 81 kB 00:00 2022-09-20T20:04:05.571 INFO:teuthology.orchestra.run.smithi036.stdout:ceph packages for x86_64 269 kB/s | 81 kB 00:00 2022-09-20T20:04:05.806 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 37 kB/s | 9.4 kB 00:00 2022-09-20T20:04:05.855 INFO:teuthology.orchestra.run.smithi036.stdout:ceph noarch packages 38 kB/s | 9.4 kB 00:00 2022-09-20T20:04:06.082 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 5.9 kB/s | 1.5 kB 00:00 2022-09-20T20:04:06.139 INFO:teuthology.orchestra.run.smithi036.stdout:ceph source packages 5.8 kB/s | 1.5 kB 00:00 2022-09-20T20:04:06.249 INFO:teuthology.orchestra.run.smithi115.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2022-09-20T20:04:06.290 INFO:teuthology.orchestra.run.smithi036.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2022-09-20T20:04:07.225 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:00 2022-09-20T20:04:07.257 INFO:teuthology.orchestra.run.smithi036.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2022-09-20T20:04:10.025 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 410 kB/s | 24 kB 00:00 2022-09-20T20:04:10.252 INFO:teuthology.orchestra.run.smithi036.stdout:lab-extras 294 kB/s | 24 kB 00:00 2022-09-20T20:04:11.001 INFO:teuthology.orchestra.run.smithi036.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 69 MB/s | 47 MB 00:00 2022-09-20T20:04:11.035 INFO:teuthology.orchestra.run.smithi115.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 55 MB/s | 52 MB 00:00 2022-09-20T20:04:18.844 INFO:teuthology.orchestra.run.smithi115.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 63 MB/s | 47 MB 00:00 2022-09-20T20:04:19.327 INFO:teuthology.orchestra.run.smithi036.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 74 MB/s | 52 MB 00:00 2022-09-20T20:04:29.879 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-09-20T20:04:29.881 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-09-20T20:04:29.894 INFO:teuthology.orchestra.run.smithi115.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-09-20T20:04:29.895 INFO:teuthology.orchestra.run.smithi115.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-09-20T20:04:29.896 INFO:teuthology.orchestra.run.smithi115.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-09-20T20:04:29.897 INFO:teuthology.orchestra.run.smithi115.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-09-20T20:04:29.900 INFO:teuthology.orchestra.run.smithi115.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-09-20T20:04:29.903 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-09-20T20:04:29.904 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-09-20T20:04:30.038 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2022-09-20T20:04:30.050 INFO:teuthology.orchestra.run.smithi115.stdout:=========================================================================================================================== 2022-09-20T20:04:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2022-09-20T20:04:30.051 INFO:teuthology.orchestra.run.smithi115.stdout:=========================================================================================================================== 2022-09-20T20:04:30.051 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2022-09-20T20:04:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-7.el8_6 rhel-8-for-x86_64-baseos-rpms 137 k 2022-09-20T20:04:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-09-20T20:04:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:17.0.0-15134.gada1013c.el8 ceph 6.4 k 2022-09-20T20:04:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:17.0.0-15134.gada1013c.el8 ceph 5.2 M 2022-09-20T20:04:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:17.0.0-15134.gada1013c.el8 ceph 843 k 2022-09-20T20:04:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:17.0.0-15134.gada1013c.el8 ceph 165 k 2022-09-20T20:04:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:17.0.0-15134.gada1013c.el8 ceph 1.5 M 2022-09-20T20:04:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 117 k 2022-09-20T20:04:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 1.6 M 2022-09-20T20:04:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 7.4 M 2022-09-20T20:04:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 47 k 2022-09-20T20:04:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:17.0.0-15134.gada1013c.el8 ceph 18 M 2022-09-20T20:04:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:17.0.0-15134.gada1013c.el8 ceph 54 M 2022-09-20T20:04:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 263 k 2022-09-20T20:04:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 81 k 2022-09-20T20:04:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror x86_64 2:17.0.0-15134.gada1013c.el8 ceph 257 k 2022-09-20T20:04:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 15 k 2022-09-20T20:04:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-09-20T20:04:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-09-20T20:04:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-09-20T20:04:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:17.0.0-15134.gada1013c.el8 ceph 30 k 2022-09-20T20:04:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:17.0.0-15134.gada1013c.el8 ceph 766 k 2022-09-20T20:04:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:17.0.0-15134.gada1013c.el8 ceph 139 k 2022-09-20T20:04:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-09-20T20:04:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-09-20T20:04:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-09-20T20:04:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-09-20T20:04:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-09-20T20:04:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-09-20T20:04:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:17.0.0-15134.gada1013c.el8 ceph 190 k 2022-09-20T20:04:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:17.0.0-15134.gada1013c.el8 ceph 371 k 2022-09-20T20:04:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:17.0.0-15134.gada1013c.el8 ceph 358 k 2022-09-20T20:04:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:17.0.0-15134.gada1013c.el8 ceph 106 k 2022-09-20T20:04:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:17.0.0-15134.gada1013c.el8 ceph 94 k 2022-09-20T20:04:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:17.0.0-15134.gada1013c.el8 ceph 3.5 M 2022-09-20T20:04:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:17.0.0-15134.gada1013c.el8 ceph 175 k 2022-09-20T20:04:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-09-20T20:04:30.061 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2022-09-20T20:04:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-09-20T20:04:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:17.0.0-15134.gada1013c.el8 ceph 3.6 M 2022-09-20T20:04:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:17.0.0-15134.gada1013c.el8 ceph 3.6 M 2022-09-20T20:04:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-09-20T20:04:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-09-20T20:04:30.062 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2022-09-20T20:04:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-09-20T20:04:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:17.0.0-15134.gada1013c.el8 ceph 24 M 2022-09-20T20:04:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 23 k 2022-09-20T20:04:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:17.0.0-15134.gada1013c.el8 ceph 2.3 M 2022-09-20T20:04:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 233 k 2022-09-20T20:04:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:17.0.0-15134.gada1013c.el8 ceph 3.9 M 2022-09-20T20:04:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:17.0.0-15134.gada1013c.el8 ceph 17 M 2022-09-20T20:04:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 14 k 2022-09-20T20:04:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:17.0.0-15134.gada1013c.el8 ceph 24 k 2022-09-20T20:04:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-09-20T20:04:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-09-20T20:04:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-09-20T20:04:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-09-20T20:04:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-09-20T20:04:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:17.0.0-15134.gada1013c.el8 ceph 137 k 2022-09-20T20:04:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-09-20T20:04:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-09-20T20:04:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-09-20T20:04:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-09-20T20:04:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-09-20T20:04:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-09-20T20:04:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:17.0.0-15134.gada1013c.el8 ceph 473 k 2022-09-20T20:04:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-09-20T20:04:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:17.0.0-15134.gada1013c.el8 ceph 6.4 M 2022-09-20T20:04:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-09-20T20:04:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-09-20T20:04:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-09-20T20:04:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-09-20T20:04:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-09-20T20:04:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-09-20T20:04:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-09-20T20:04:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-09-20T20:04:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-09-20T20:04:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-09-20T20:04:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-09-20T20:04:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-09-20T20:04:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-09-20T20:04:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-09-20T20:04:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-09-20T20:04:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-09-20T20:04:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:17.0.0-15134.gada1013c.el8 ceph 45 k 2022-09-20T20:04:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:17.0.0-15134.gada1013c.el8 ceph 109 k 2022-09-20T20:04:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-09-20T20:04:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-09-20T20:04:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-09-20T20:04:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-09-20T20:04:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-09-20T20:04:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-09-20T20:04:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-09-20T20:04:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-09-20T20:04:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-09-20T20:04:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-09-20T20:04:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-09-20T20:04:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-09-20T20:04:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-09-20T20:04:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-09-20T20:04:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-09-20T20:04:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-09-20T20:04:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-09-20T20:04:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-09-20T20:04:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-09-20T20:04:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-09-20T20:04:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-09-20T20:04:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-09-20T20:04:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-09-20T20:04:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-09-20T20:04:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-09-20T20:04:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-09-20T20:04:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-09-20T20:04:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-09-20T20:04:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-09-20T20:04:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-09-20T20:04:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-09-20T20:04:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-09-20T20:04:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-09-20T20:04:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-09-20T20:04:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-09-20T20:04:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-09-20T20:04:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-09-20T20:04:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-09-20T20:04:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-09-20T20:04:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-09-20T20:04:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-09-20T20:04:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-09-20T20:04:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-09-20T20:04:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-09-20T20:04:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-09-20T20:04:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-09-20T20:04:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-09-20T20:04:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-09-20T20:04:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-09-20T20:04:30.085 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling module streams: 2022-09-20T20:04:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql 10 2022-09-20T20:04:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:04:30.086 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2022-09-20T20:04:30.086 INFO:teuthology.orchestra.run.smithi115.stdout:=========================================================================================================================== 2022-09-20T20:04:30.086 INFO:teuthology.orchestra.run.smithi115.stdout:Install 128 Packages 2022-09-20T20:04:30.086 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 5 Packages 2022-09-20T20:04:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:04:30.087 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 218 M 2022-09-20T20:04:30.087 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2022-09-20T20:04:30.343 INFO:teuthology.orchestra.run.smithi115.stdout:(1/133): ceph-17.0.0-15134.gada1013c.el8.x86_64 23 kB/s | 6.4 kB 00:00 2022-09-20T20:04:30.931 INFO:teuthology.orchestra.run.smithi036.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-09-20T20:04:30.932 INFO:teuthology.orchestra.run.smithi036.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-09-20T20:04:30.947 INFO:teuthology.orchestra.run.smithi036.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-09-20T20:04:30.949 INFO:teuthology.orchestra.run.smithi036.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-09-20T20:04:30.950 INFO:teuthology.orchestra.run.smithi036.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-09-20T20:04:30.951 INFO:teuthology.orchestra.run.smithi036.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-09-20T20:04:30.953 INFO:teuthology.orchestra.run.smithi036.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-09-20T20:04:30.957 INFO:teuthology.orchestra.run.smithi036.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-09-20T20:04:30.958 INFO:teuthology.orchestra.run.smithi036.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-09-20T20:04:31.106 INFO:teuthology.orchestra.run.smithi036.stdout:Dependencies resolved. 2022-09-20T20:04:31.120 INFO:teuthology.orchestra.run.smithi036.stdout:=========================================================================================================================== 2022-09-20T20:04:31.120 INFO:teuthology.orchestra.run.smithi036.stdout: Package Arch Version Repository Size 2022-09-20T20:04:31.121 INFO:teuthology.orchestra.run.smithi036.stdout:=========================================================================================================================== 2022-09-20T20:04:31.121 INFO:teuthology.orchestra.run.smithi036.stdout:Installing: 2022-09-20T20:04:31.121 INFO:teuthology.orchestra.run.smithi036.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-7.el8_6 rhel-8-for-x86_64-baseos-rpms 137 k 2022-09-20T20:04:31.121 INFO:teuthology.orchestra.run.smithi036.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-09-20T20:04:31.122 INFO:teuthology.orchestra.run.smithi036.stdout: ceph x86_64 2:17.0.0-15134.gada1013c.el8 ceph 6.4 k 2022-09-20T20:04:31.122 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-base x86_64 2:17.0.0-15134.gada1013c.el8 ceph 5.2 M 2022-09-20T20:04:31.122 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-fuse x86_64 2:17.0.0-15134.gada1013c.el8 ceph 843 k 2022-09-20T20:04:31.122 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-immutable-object-cache x86_64 2:17.0.0-15134.gada1013c.el8 ceph 165 k 2022-09-20T20:04:31.123 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr x86_64 2:17.0.0-15134.gada1013c.el8 ceph 1.5 M 2022-09-20T20:04:31.123 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-cephadm noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 117 k 2022-09-20T20:04:31.123 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-dashboard noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 1.6 M 2022-09-20T20:04:31.123 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-diskprediction-local noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 7.4 M 2022-09-20T20:04:31.124 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-rook noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 47 k 2022-09-20T20:04:31.124 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-radosgw x86_64 2:17.0.0-15134.gada1013c.el8 ceph 18 M 2022-09-20T20:04:31.124 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-test x86_64 2:17.0.0-15134.gada1013c.el8 ceph 54 M 2022-09-20T20:04:31.124 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-volume noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 263 k 2022-09-20T20:04:31.125 INFO:teuthology.orchestra.run.smithi036.stdout: cephadm noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 81 k 2022-09-20T20:04:31.125 INFO:teuthology.orchestra.run.smithi036.stdout: cephfs-mirror x86_64 2:17.0.0-15134.gada1013c.el8 ceph 257 k 2022-09-20T20:04:31.126 INFO:teuthology.orchestra.run.smithi036.stdout: cephfs-top noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 15 k 2022-09-20T20:04:31.126 INFO:teuthology.orchestra.run.smithi036.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-09-20T20:04:31.126 INFO:teuthology.orchestra.run.smithi036.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-09-20T20:04:31.126 INFO:teuthology.orchestra.run.smithi036.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-09-20T20:04:31.127 INFO:teuthology.orchestra.run.smithi036.stdout: libcephfs-devel x86_64 2:17.0.0-15134.gada1013c.el8 ceph 30 k 2022-09-20T20:04:31.127 INFO:teuthology.orchestra.run.smithi036.stdout: libcephfs2 x86_64 2:17.0.0-15134.gada1013c.el8 ceph 766 k 2022-09-20T20:04:31.127 INFO:teuthology.orchestra.run.smithi036.stdout: librados-devel x86_64 2:17.0.0-15134.gada1013c.el8 ceph 139 k 2022-09-20T20:04:31.127 INFO:teuthology.orchestra.run.smithi036.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-09-20T20:04:31.127 INFO:teuthology.orchestra.run.smithi036.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-09-20T20:04:31.128 INFO:teuthology.orchestra.run.smithi036.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-09-20T20:04:31.128 INFO:teuthology.orchestra.run.smithi036.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-09-20T20:04:31.128 INFO:teuthology.orchestra.run.smithi036.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-09-20T20:04:31.128 INFO:teuthology.orchestra.run.smithi036.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-09-20T20:04:31.129 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cephfs x86_64 2:17.0.0-15134.gada1013c.el8 ceph 190 k 2022-09-20T20:04:31.129 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rados x86_64 2:17.0.0-15134.gada1013c.el8 ceph 371 k 2022-09-20T20:04:31.129 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rbd x86_64 2:17.0.0-15134.gada1013c.el8 ceph 358 k 2022-09-20T20:04:31.129 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rgw x86_64 2:17.0.0-15134.gada1013c.el8 ceph 106 k 2022-09-20T20:04:31.129 INFO:teuthology.orchestra.run.smithi036.stdout: rbd-fuse x86_64 2:17.0.0-15134.gada1013c.el8 ceph 94 k 2022-09-20T20:04:31.130 INFO:teuthology.orchestra.run.smithi036.stdout: rbd-mirror x86_64 2:17.0.0-15134.gada1013c.el8 ceph 3.5 M 2022-09-20T20:04:31.130 INFO:teuthology.orchestra.run.smithi036.stdout: rbd-nbd x86_64 2:17.0.0-15134.gada1013c.el8 ceph 175 k 2022-09-20T20:04:31.130 INFO:teuthology.orchestra.run.smithi036.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-09-20T20:04:31.130 INFO:teuthology.orchestra.run.smithi036.stdout:Upgrading: 2022-09-20T20:04:31.131 INFO:teuthology.orchestra.run.smithi036.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-09-20T20:04:31.131 INFO:teuthology.orchestra.run.smithi036.stdout: librados2 x86_64 2:17.0.0-15134.gada1013c.el8 ceph 3.6 M 2022-09-20T20:04:31.131 INFO:teuthology.orchestra.run.smithi036.stdout: librbd1 x86_64 2:17.0.0-15134.gada1013c.el8 ceph 3.6 M 2022-09-20T20:04:31.131 INFO:teuthology.orchestra.run.smithi036.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-09-20T20:04:31.131 INFO:teuthology.orchestra.run.smithi036.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-09-20T20:04:31.132 INFO:teuthology.orchestra.run.smithi036.stdout:Installing dependencies: 2022-09-20T20:04:31.132 INFO:teuthology.orchestra.run.smithi036.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-09-20T20:04:31.132 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-common x86_64 2:17.0.0-15134.gada1013c.el8 ceph 24 M 2022-09-20T20:04:31.132 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-grafana-dashboards noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 23 k 2022-09-20T20:04:31.132 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mds x86_64 2:17.0.0-15134.gada1013c.el8 ceph 2.3 M 2022-09-20T20:04:31.133 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-modules-core noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 233 k 2022-09-20T20:04:31.133 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mon x86_64 2:17.0.0-15134.gada1013c.el8 ceph 3.9 M 2022-09-20T20:04:31.133 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-osd x86_64 2:17.0.0-15134.gada1013c.el8 ceph 17 M 2022-09-20T20:04:31.133 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-prometheus-alerts noarch 2:17.0.0-15134.gada1013c.el8 ceph-noarch 14 k 2022-09-20T20:04:31.134 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-selinux x86_64 2:17.0.0-15134.gada1013c.el8 ceph 24 k 2022-09-20T20:04:31.134 INFO:teuthology.orchestra.run.smithi036.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-09-20T20:04:31.134 INFO:teuthology.orchestra.run.smithi036.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-09-20T20:04:31.134 INFO:teuthology.orchestra.run.smithi036.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-09-20T20:04:31.134 INFO:teuthology.orchestra.run.smithi036.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-09-20T20:04:31.135 INFO:teuthology.orchestra.run.smithi036.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-09-20T20:04:31.135 INFO:teuthology.orchestra.run.smithi036.stdout: libcephsqlite x86_64 2:17.0.0-15134.gada1013c.el8 ceph 137 k 2022-09-20T20:04:31.135 INFO:teuthology.orchestra.run.smithi036.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-09-20T20:04:31.135 INFO:teuthology.orchestra.run.smithi036.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-09-20T20:04:31.136 INFO:teuthology.orchestra.run.smithi036.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-09-20T20:04:31.136 INFO:teuthology.orchestra.run.smithi036.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-09-20T20:04:31.136 INFO:teuthology.orchestra.run.smithi036.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-09-20T20:04:31.136 INFO:teuthology.orchestra.run.smithi036.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-09-20T20:04:31.136 INFO:teuthology.orchestra.run.smithi036.stdout: libradosstriper1 x86_64 2:17.0.0-15134.gada1013c.el8 ceph 473 k 2022-09-20T20:04:31.137 INFO:teuthology.orchestra.run.smithi036.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-09-20T20:04:31.137 INFO:teuthology.orchestra.run.smithi036.stdout: librgw2 x86_64 2:17.0.0-15134.gada1013c.el8 ceph 6.4 M 2022-09-20T20:04:31.137 INFO:teuthology.orchestra.run.smithi036.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-09-20T20:04:31.137 INFO:teuthology.orchestra.run.smithi036.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-09-20T20:04:31.137 INFO:teuthology.orchestra.run.smithi036.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-09-20T20:04:31.138 INFO:teuthology.orchestra.run.smithi036.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-09-20T20:04:31.138 INFO:teuthology.orchestra.run.smithi036.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-09-20T20:04:31.138 INFO:teuthology.orchestra.run.smithi036.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-09-20T20:04:31.138 INFO:teuthology.orchestra.run.smithi036.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-09-20T20:04:31.139 INFO:teuthology.orchestra.run.smithi036.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-09-20T20:04:31.139 INFO:teuthology.orchestra.run.smithi036.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-09-20T20:04:31.139 INFO:teuthology.orchestra.run.smithi036.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-09-20T20:04:31.139 INFO:teuthology.orchestra.run.smithi036.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-09-20T20:04:31.139 INFO:teuthology.orchestra.run.smithi036.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-09-20T20:04:31.140 INFO:teuthology.orchestra.run.smithi036.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-09-20T20:04:31.140 INFO:teuthology.orchestra.run.smithi036.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-09-20T20:04:31.140 INFO:teuthology.orchestra.run.smithi036.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-09-20T20:04:31.140 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-09-20T20:04:31.140 INFO:teuthology.orchestra.run.smithi036.stdout: python3-ceph-argparse x86_64 2:17.0.0-15134.gada1013c.el8 ceph 45 k 2022-09-20T20:04:31.141 INFO:teuthology.orchestra.run.smithi036.stdout: python3-ceph-common x86_64 2:17.0.0-15134.gada1013c.el8 ceph 109 k 2022-09-20T20:04:31.141 INFO:teuthology.orchestra.run.smithi036.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-09-20T20:04:31.141 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-09-20T20:04:31.141 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-09-20T20:04:31.141 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-09-20T20:04:31.142 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-09-20T20:04:31.142 INFO:teuthology.orchestra.run.smithi036.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-09-20T20:04:31.142 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-09-20T20:04:31.142 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-09-20T20:04:31.143 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-09-20T20:04:31.143 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-09-20T20:04:31.143 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-09-20T20:04:31.143 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-09-20T20:04:31.143 INFO:teuthology.orchestra.run.smithi036.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-09-20T20:04:31.144 INFO:teuthology.orchestra.run.smithi036.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-09-20T20:04:31.144 INFO:teuthology.orchestra.run.smithi036.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-09-20T20:04:31.144 INFO:teuthology.orchestra.run.smithi036.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-09-20T20:04:31.144 INFO:teuthology.orchestra.run.smithi036.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-09-20T20:04:31.145 INFO:teuthology.orchestra.run.smithi036.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-09-20T20:04:31.145 INFO:teuthology.orchestra.run.smithi036.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-09-20T20:04:31.145 INFO:teuthology.orchestra.run.smithi036.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-09-20T20:04:31.145 INFO:teuthology.orchestra.run.smithi036.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-09-20T20:04:31.145 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-09-20T20:04:31.146 INFO:teuthology.orchestra.run.smithi036.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-09-20T20:04:31.146 INFO:teuthology.orchestra.run.smithi036.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-09-20T20:04:31.146 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-09-20T20:04:31.146 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-09-20T20:04:31.147 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-09-20T20:04:31.147 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-09-20T20:04:31.147 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-09-20T20:04:31.147 INFO:teuthology.orchestra.run.smithi036.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-09-20T20:04:31.148 INFO:teuthology.orchestra.run.smithi036.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-09-20T20:04:31.148 INFO:teuthology.orchestra.run.smithi036.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-09-20T20:04:31.148 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-09-20T20:04:31.148 INFO:teuthology.orchestra.run.smithi036.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-09-20T20:04:31.148 INFO:teuthology.orchestra.run.smithi036.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-09-20T20:04:31.149 INFO:teuthology.orchestra.run.smithi036.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-09-20T20:04:31.149 INFO:teuthology.orchestra.run.smithi036.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-09-20T20:04:31.149 INFO:teuthology.orchestra.run.smithi036.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-09-20T20:04:31.149 INFO:teuthology.orchestra.run.smithi036.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-09-20T20:04:31.149 INFO:teuthology.orchestra.run.smithi036.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-09-20T20:04:31.150 INFO:teuthology.orchestra.run.smithi036.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-09-20T20:04:31.150 INFO:teuthology.orchestra.run.smithi036.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-09-20T20:04:31.150 INFO:teuthology.orchestra.run.smithi036.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-09-20T20:04:31.150 INFO:teuthology.orchestra.run.smithi036.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-09-20T20:04:31.151 INFO:teuthology.orchestra.run.smithi036.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-09-20T20:04:31.151 INFO:teuthology.orchestra.run.smithi036.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-09-20T20:04:31.151 INFO:teuthology.orchestra.run.smithi036.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-09-20T20:04:31.151 INFO:teuthology.orchestra.run.smithi036.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-09-20T20:04:31.152 INFO:teuthology.orchestra.run.smithi036.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-09-20T20:04:31.152 INFO:teuthology.orchestra.run.smithi036.stdout:Enabling module streams: 2022-09-20T20:04:31.152 INFO:teuthology.orchestra.run.smithi036.stdout: postgresql 10 2022-09-20T20:04:31.152 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:04:31.153 INFO:teuthology.orchestra.run.smithi036.stdout:Transaction Summary 2022-09-20T20:04:31.153 INFO:teuthology.orchestra.run.smithi036.stdout:=========================================================================================================================== 2022-09-20T20:04:31.153 INFO:teuthology.orchestra.run.smithi036.stdout:Install 128 Packages 2022-09-20T20:04:31.153 INFO:teuthology.orchestra.run.smithi036.stdout:Upgrade 5 Packages 2022-09-20T20:04:31.154 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:04:31.154 INFO:teuthology.orchestra.run.smithi036.stdout:Total download size: 218 M 2022-09-20T20:04:31.155 INFO:teuthology.orchestra.run.smithi036.stdout:Downloading Packages: 2022-09-20T20:04:31.235 INFO:teuthology.orchestra.run.smithi115.stdout:(2/133): ceph-fuse-17.0.0-15134.gada1013c.el8.x 945 kB/s | 843 kB 00:00 2022-09-20T20:04:31.294 INFO:teuthology.orchestra.run.smithi036.stdout:(1/133): ceph-17.0.0-15134.gada1013c.el8.x86_64 41 kB/s | 6.4 kB 00:00 2022-09-20T20:04:31.377 INFO:teuthology.orchestra.run.smithi115.stdout:(3/133): ceph-immutable-object-cache-17.0.0-151 1.1 MB/s | 165 kB 00:00 2022-09-20T20:04:32.044 INFO:teuthology.orchestra.run.smithi115.stdout:(4/133): ceph-mds-17.0.0-15134.gada1013c.el8.x8 3.4 MB/s | 2.3 MB 00:00 2022-09-20T20:04:32.169 INFO:teuthology.orchestra.run.smithi036.stdout:(2/133): ceph-fuse-17.0.0-15134.gada1013c.el8.x 964 kB/s | 843 kB 00:00 2022-09-20T20:04:32.186 INFO:teuthology.orchestra.run.smithi115.stdout:(5/133): ceph-base-17.0.0-15134.gada1013c.el8.x 2.4 MB/s | 5.2 MB 00:02 2022-09-20T20:04:32.286 INFO:teuthology.orchestra.run.smithi036.stdout:(3/133): ceph-immutable-object-cache-17.0.0-151 1.4 MB/s | 165 kB 00:00 2022-09-20T20:04:32.403 INFO:teuthology.orchestra.run.smithi115.stdout:(6/133): ceph-mgr-17.0.0-15134.gada1013c.el8.x8 4.3 MB/s | 1.5 MB 00:00 2022-09-20T20:04:32.770 INFO:teuthology.orchestra.run.smithi115.stdout:(7/133): ceph-mon-17.0.0-15134.gada1013c.el8.x8 6.6 MB/s | 3.9 MB 00:00 2022-09-20T20:04:33.070 INFO:teuthology.orchestra.run.smithi036.stdout:(4/133): ceph-base-17.0.0-15134.gada1013c.el8.x 2.7 MB/s | 5.2 MB 00:01 2022-09-20T20:04:33.253 INFO:teuthology.orchestra.run.smithi036.stdout:(5/133): ceph-mds-17.0.0-15134.gada1013c.el8.x8 2.3 MB/s | 2.3 MB 00:00 2022-09-20T20:04:33.762 INFO:teuthology.orchestra.run.smithi036.stdout:(6/133): ceph-mon-17.0.0-15134.gada1013c.el8.x8 7.6 MB/s | 3.9 MB 00:00 2022-09-20T20:04:34.754 INFO:teuthology.orchestra.run.smithi036.stdout:(7/133): ceph-osd-17.0.0-15134.gada1013c.el8.x8 17 MB/s | 17 MB 00:00 2022-09-20T20:04:35.279 INFO:teuthology.orchestra.run.smithi036.stdout:(8/133): ceph-mgr-17.0.0-15134.gada1013c.el8.x8 709 kB/s | 1.5 MB 00:02 2022-09-20T20:04:35.388 INFO:teuthology.orchestra.run.smithi036.stdout:(9/133): ceph-selinux-17.0.0-15134.gada1013c.el 224 kB/s | 24 kB 00:00 2022-09-20T20:04:36.313 INFO:teuthology.orchestra.run.smithi036.stdout:(10/133): ceph-radosgw-17.0.0-15134.gada1013c.e 11 MB/s | 18 MB 00:01 2022-09-20T20:04:36.369 INFO:teuthology.orchestra.run.smithi115.stdout:(8/133): ceph-common-17.0.0-15134.gada1013c.el8 3.8 MB/s | 24 MB 00:06 2022-09-20T20:04:36.455 INFO:teuthology.orchestra.run.smithi036.stdout:(11/133): cephfs-mirror-17.0.0-15134.gada1013c. 1.8 MB/s | 257 kB 00:00 2022-09-20T20:04:36.461 INFO:teuthology.orchestra.run.smithi115.stdout:(9/133): ceph-selinux-17.0.0-15134.gada1013c.el 265 kB/s | 24 kB 00:00 2022-09-20T20:04:36.522 INFO:teuthology.orchestra.run.smithi036.stdout:(12/133): libcephfs-devel-17.0.0-15134.gada1013 459 kB/s | 30 kB 00:00 2022-09-20T20:04:36.681 INFO:teuthology.orchestra.run.smithi036.stdout:(13/133): libcephfs2-17.0.0-15134.gada1013c.el8 4.7 MB/s | 766 kB 00:00 2022-09-20T20:04:36.798 INFO:teuthology.orchestra.run.smithi036.stdout:(14/133): libcephsqlite-17.0.0-15134.gada1013c. 1.2 MB/s | 137 kB 00:00 2022-09-20T20:04:37.015 INFO:teuthology.orchestra.run.smithi036.stdout:(15/133): librados-devel-17.0.0-15134.gada1013c 641 kB/s | 139 kB 00:00 2022-09-20T20:04:37.157 INFO:teuthology.orchestra.run.smithi036.stdout:(16/133): libradosstriper1-17.0.0-15134.gada101 3.3 MB/s | 473 kB 00:00 2022-09-20T20:04:37.520 INFO:teuthology.orchestra.run.smithi115.stdout:(10/133): ceph-radosgw-17.0.0-15134.gada1013c.e 3.8 MB/s | 18 MB 00:04 2022-09-20T20:04:37.695 INFO:teuthology.orchestra.run.smithi115.stdout:(11/133): cephfs-mirror-17.0.0-15134.gada1013c. 1.4 MB/s | 257 kB 00:00 2022-09-20T20:04:37.753 INFO:teuthology.orchestra.run.smithi115.stdout:(12/133): libcephfs-devel-17.0.0-15134.gada1013 524 kB/s | 30 kB 00:00 2022-09-20T20:04:37.782 INFO:teuthology.orchestra.run.smithi036.stdout:(17/133): librgw2-17.0.0-15134.gada1013c.el8.x8 10 MB/s | 6.4 MB 00:00 2022-09-20T20:04:37.865 INFO:teuthology.orchestra.run.smithi036.stdout:(18/133): python3-ceph-argparse-17.0.0-15134.ga 537 kB/s | 45 kB 00:00 2022-09-20T20:04:37.920 INFO:teuthology.orchestra.run.smithi115.stdout:(13/133): libcephfs2-17.0.0-15134.gada1013c.el8 4.5 MB/s | 766 kB 00:00 2022-09-20T20:04:37.949 INFO:teuthology.orchestra.run.smithi036.stdout:(19/133): python3-ceph-common-17.0.0-15134.gada 1.3 MB/s | 109 kB 00:00 2022-09-20T20:04:37.996 INFO:teuthology.orchestra.run.smithi115.stdout:(14/133): libcephsqlite-17.0.0-15134.gada1013c. 1.8 MB/s | 137 kB 00:00 2022-09-20T20:04:38.088 INFO:teuthology.orchestra.run.smithi115.stdout:(15/133): librados-devel-17.0.0-15134.gada1013c 1.5 MB/s | 139 kB 00:00 2022-09-20T20:04:38.116 INFO:teuthology.orchestra.run.smithi036.stdout:(20/133): python3-cephfs-17.0.0-15134.gada1013c 1.1 MB/s | 190 kB 00:00 2022-09-20T20:04:38.213 INFO:teuthology.orchestra.run.smithi115.stdout:(16/133): libradosstriper1-17.0.0-15134.gada101 3.7 MB/s | 473 kB 00:00 2022-09-20T20:04:38.233 INFO:teuthology.orchestra.run.smithi036.stdout:(21/133): python3-rados-17.0.0-15134.gada1013c. 3.1 MB/s | 371 kB 00:00 2022-09-20T20:04:38.367 INFO:teuthology.orchestra.run.smithi036.stdout:(22/133): python3-rbd-17.0.0-15134.gada1013c.el 2.6 MB/s | 358 kB 00:00 2022-09-20T20:04:38.509 INFO:teuthology.orchestra.run.smithi036.stdout:(23/133): python3-rgw-17.0.0-15134.gada1013c.el 752 kB/s | 106 kB 00:00 2022-09-20T20:04:38.609 INFO:teuthology.orchestra.run.smithi036.stdout:(24/133): rbd-fuse-17.0.0-15134.gada1013c.el8.x 943 kB/s | 94 kB 00:00 2022-09-20T20:04:39.057 INFO:teuthology.orchestra.run.smithi036.stdout:(25/133): rbd-mirror-17.0.0-15134.gada1013c.el8 7.8 MB/s | 3.5 MB 00:00 2022-09-20T20:04:39.160 INFO:teuthology.orchestra.run.smithi036.stdout:(26/133): rbd-nbd-17.0.0-15134.gada1013c.el8.x8 1.7 MB/s | 175 kB 00:00 2022-09-20T20:04:39.246 INFO:teuthology.orchestra.run.smithi115.stdout:(17/133): ceph-osd-17.0.0-15134.gada1013c.el8.x 2.5 MB/s | 17 MB 00:06 2022-09-20T20:04:39.268 INFO:teuthology.orchestra.run.smithi036.stdout:(27/133): ceph-grafana-dashboards-17.0.0-15134. 211 kB/s | 23 kB 00:00 2022-09-20T20:04:39.330 INFO:teuthology.orchestra.run.smithi115.stdout:(18/133): librgw2-17.0.0-15134.gada1013c.el8.x8 5.7 MB/s | 6.4 MB 00:01 2022-09-20T20:04:39.344 INFO:teuthology.orchestra.run.smithi036.stdout:(28/133): ceph-mgr-cephadm-17.0.0-15134.gada101 1.5 MB/s | 117 kB 00:00 2022-09-20T20:04:39.355 INFO:teuthology.orchestra.run.smithi115.stdout:(19/133): python3-ceph-argparse-17.0.0-15134.ga 411 kB/s | 45 kB 00:00 2022-09-20T20:04:39.414 INFO:teuthology.orchestra.run.smithi115.stdout:(20/133): python3-ceph-common-17.0.0-15134.gada 1.3 MB/s | 109 kB 00:00 2022-09-20T20:04:39.439 INFO:teuthology.orchestra.run.smithi115.stdout:(21/133): python3-cephfs-17.0.0-15134.gada1013c 2.2 MB/s | 190 kB 00:00 2022-09-20T20:04:39.581 INFO:teuthology.orchestra.run.smithi115.stdout:(22/133): python3-rados-17.0.0-15134.gada1013c. 2.2 MB/s | 371 kB 00:00 2022-09-20T20:04:39.606 INFO:teuthology.orchestra.run.smithi115.stdout:(23/133): python3-rbd-17.0.0-15134.gada1013c.el 2.1 MB/s | 358 kB 00:00 2022-09-20T20:04:39.619 INFO:teuthology.orchestra.run.smithi036.stdout:(29/133): ceph-mgr-dashboard-17.0.0-15134.gada1 6.0 MB/s | 1.6 MB 00:00 2022-09-20T20:04:39.631 INFO:teuthology.orchestra.run.smithi115.stdout:(24/133): python3-rgw-17.0.0-15134.gada1013c.el 2.1 MB/s | 106 kB 00:00 2022-09-20T20:04:39.665 INFO:teuthology.orchestra.run.smithi115.stdout:(25/133): rbd-fuse-17.0.0-15134.gada1013c.el8.x 1.6 MB/s | 94 kB 00:00 2022-09-20T20:04:39.765 INFO:teuthology.orchestra.run.smithi115.stdout:(26/133): rbd-nbd-17.0.0-15134.gada1013c.el8.x8 1.7 MB/s | 175 kB 00:00 2022-09-20T20:04:39.857 INFO:teuthology.orchestra.run.smithi115.stdout:(27/133): ceph-grafana-dashboards-17.0.0-15134. 249 kB/s | 23 kB 00:00 2022-09-20T20:04:39.911 INFO:teuthology.orchestra.run.smithi036.stdout:(30/133): ceph-common-17.0.0-15134.gada1013c.el 2.7 MB/s | 24 MB 00:08 2022-09-20T20:04:39.933 INFO:teuthology.orchestra.run.smithi115.stdout:(28/133): ceph-mgr-cephadm-17.0.0-15134.gada101 1.5 MB/s | 117 kB 00:00 2022-09-20T20:04:40.053 INFO:teuthology.orchestra.run.smithi036.stdout:(31/133): ceph-mgr-modules-core-17.0.0-15134.ga 1.6 MB/s | 233 kB 00:00 2022-09-20T20:04:40.153 INFO:teuthology.orchestra.run.smithi036.stdout:(32/133): ceph-mgr-rook-17.0.0-15134.gada1013c. 470 kB/s | 47 kB 00:00 2022-09-20T20:04:40.212 INFO:teuthology.orchestra.run.smithi036.stdout:(33/133): ceph-prometheus-alerts-17.0.0-15134.g 246 kB/s | 14 kB 00:00 2022-09-20T20:04:40.216 INFO:teuthology.orchestra.run.smithi115.stdout:(29/133): rbd-mirror-17.0.0-15134.gada1013c.el8 5.9 MB/s | 3.5 MB 00:00 2022-09-20T20:04:40.337 INFO:teuthology.orchestra.run.smithi036.stdout:(34/133): ceph-volume-17.0.0-15134.gada1013c.el 2.1 MB/s | 263 kB 00:00 2022-09-20T20:04:40.429 INFO:teuthology.orchestra.run.smithi036.stdout:(35/133): cephadm-17.0.0-15134.gada1013c.el8.no 886 kB/s | 81 kB 00:00 2022-09-20T20:04:40.508 INFO:teuthology.orchestra.run.smithi115.stdout:(30/133): ceph-mgr-dashboard-17.0.0-15134.gada1 2.9 MB/s | 1.6 MB 00:00 2022-09-20T20:04:40.521 INFO:teuthology.orchestra.run.smithi036.stdout:(36/133): cephfs-top-17.0.0-15134.gada1013c.el8 169 kB/s | 15 kB 00:00 2022-09-20T20:04:40.684 INFO:teuthology.orchestra.run.smithi115.stdout:(31/133): ceph-mgr-modules-core-17.0.0-15134.ga 1.3 MB/s | 233 kB 00:00 2022-09-20T20:04:40.776 INFO:teuthology.orchestra.run.smithi115.stdout:(32/133): ceph-mgr-rook-17.0.0-15134.gada1013c. 518 kB/s | 47 kB 00:00 2022-09-20T20:04:40.842 INFO:teuthology.orchestra.run.smithi115.stdout:(33/133): ceph-prometheus-alerts-17.0.0-15134.g 216 kB/s | 14 kB 00:00 2022-09-20T20:04:40.880 INFO:teuthology.orchestra.run.smithi036.stdout:(37/133): ceph-mgr-diskprediction-local-17.0.0- 5.9 MB/s | 7.4 MB 00:01 2022-09-20T20:04:40.930 INFO:teuthology.orchestra.run.smithi036.stdout:(38/133): gperftools-libs-2.7-9.el8.x86_64.rpm 750 kB/s | 306 kB 00:00 2022-09-20T20:04:40.959 INFO:teuthology.orchestra.run.smithi115.stdout:(34/133): ceph-volume-17.0.0-15134.gada1013c.el 2.2 MB/s | 263 kB 00:00 2022-09-20T20:04:41.030 INFO:teuthology.orchestra.run.smithi036.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 751 kB/s | 75 kB 00:00 2022-09-20T20:04:41.035 INFO:teuthology.orchestra.run.smithi115.stdout:(35/133): cephadm-17.0.0-15134.gada1013c.el8.no 1.1 MB/s | 81 kB 00:00 2022-09-20T20:04:41.089 INFO:teuthology.orchestra.run.smithi036.stdout:(40/133): liboath-2.6.2-3.el8.x86_64.rpm 282 kB/s | 59 kB 00:00 2022-09-20T20:04:41.093 INFO:teuthology.orchestra.run.smithi115.stdout:(36/133): cephfs-top-17.0.0-15134.gada1013c.el8 265 kB/s | 15 kB 00:00 2022-09-20T20:04:41.148 INFO:teuthology.orchestra.run.smithi036.stdout:(41/133): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2022-09-20T20:04:41.173 INFO:teuthology.orchestra.run.smithi036.stdout:(42/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 521 kB/s | 44 kB 00:00 2022-09-20T20:04:41.294 INFO:teuthology.orchestra.run.smithi115.stdout:(37/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.5 MB/s | 306 kB 00:00 2022-09-20T20:04:41.315 INFO:teuthology.orchestra.run.smithi036.stdout:(43/133): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2022-09-20T20:04:41.373 INFO:teuthology.orchestra.run.smithi036.stdout:(44/133): python3-cachetools-3.1.1-4.el8.noarch 165 kB/s | 33 kB 00:00 2022-09-20T20:04:41.394 INFO:teuthology.orchestra.run.smithi115.stdout:(38/133): ceph-mgr-diskprediction-local-17.0.0- 6.3 MB/s | 7.4 MB 00:01 2022-09-20T20:04:41.399 INFO:teuthology.orchestra.run.smithi036.stdout:(45/133): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2022-09-20T20:04:41.419 INFO:teuthology.orchestra.run.smithi115.stdout:(39/133): liboath-2.6.2-3.el8.x86_64.rpm 470 kB/s | 59 kB 00:00 2022-09-20T20:04:41.503 INFO:teuthology.orchestra.run.smithi115.stdout:(40/133): python3-asyncssh-2.7.0-2.el8.noarch.r 5.6 MB/s | 476 kB 00:00 2022-09-20T20:04:41.524 INFO:teuthology.orchestra.run.smithi036.stdout:(46/133): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2022-09-20T20:04:41.528 INFO:teuthology.orchestra.run.smithi115.stdout:(41/133): libunwind-1.3.1-3.el8.x86_64.rpm 562 kB/s | 75 kB 00:00 2022-09-20T20:04:41.541 INFO:teuthology.orchestra.run.smithi036.stdout:(47/133): python3-cheroot-8.5.2-1.el8.noarch.rp 1.0 MB/s | 173 kB 00:00 2022-09-20T20:04:41.545 INFO:teuthology.orchestra.run.smithi115.stdout:(42/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2022-09-20T20:04:41.583 INFO:teuthology.orchestra.run.smithi036.stdout:(48/133): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2022-09-20T20:04:41.587 INFO:teuthology.orchestra.run.smithi115.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 790 kB/s | 33 kB 00:00 2022-09-20T20:04:41.604 INFO:teuthology.orchestra.run.smithi115.stdout:(44/133): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2022-09-20T20:04:41.617 INFO:teuthology.orchestra.run.smithi036.stdout:(49/133): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2022-09-20T20:04:41.662 INFO:teuthology.orchestra.run.smithi115.stdout:(45/133): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2022-09-20T20:04:41.688 INFO:teuthology.orchestra.run.smithi115.stdout:(46/133): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2022-09-20T20:04:41.738 INFO:teuthology.orchestra.run.smithi115.stdout:(47/133): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2022-09-20T20:04:41.763 INFO:teuthology.orchestra.run.smithi115.stdout:(48/133): python3-cherrypy-18.4.0-1.el8.noarch. 3.7 MB/s | 384 kB 00:00 2022-09-20T20:04:41.789 INFO:teuthology.orchestra.run.smithi115.stdout:(49/133): python3-jaraco-6.2-6.el8.noarch.rpm 220 kB/s | 11 kB 00:00 2022-09-20T20:04:41.792 INFO:teuthology.orchestra.run.smithi036.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 84 kB/s | 18 kB 00:00 2022-09-20T20:04:41.814 INFO:teuthology.orchestra.run.smithi115.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 349 kB/s | 18 kB 00:00 2022-09-20T20:04:41.892 INFO:teuthology.orchestra.run.smithi036.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 487 kB/s | 49 kB 00:00 2022-09-20T20:04:41.951 INFO:teuthology.orchestra.run.smithi036.stdout:(52/133): python3-kubernetes-11.0.0-6.el8.noarc 3.5 MB/s | 1.2 MB 00:00 2022-09-20T20:04:41.972 INFO:teuthology.orchestra.run.smithi115.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 307 kB/s | 49 kB 00:00 2022-09-20T20:04:41.976 INFO:teuthology.orchestra.run.smithi036.stdout:(53/133): python3-more-itertools-7.2.0-3.el8.no 703 kB/s | 59 kB 00:00 2022-09-20T20:04:42.010 INFO:teuthology.orchestra.run.smithi036.stdout:(54/133): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2022-09-20T20:04:42.190 INFO:teuthology.orchestra.run.smithi115.stdout:(52/133): python3-kubernetes-11.0.0-6.el8.noarc 2.9 MB/s | 1.2 MB 00:00 2022-09-20T20:04:42.229 INFO:teuthology.orchestra.run.smithi036.stdout:(55/133): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2022-09-20T20:04:42.231 INFO:teuthology.orchestra.run.smithi115.stdout:(53/133): python3-more-itertools-7.2.0-3.el8.no 227 kB/s | 59 kB 00:00 2022-09-20T20:04:42.244 INFO:teuthology.orchestra.run.smithi036.stdout:(56/133): python3-portend-2.6-1.el8.noarch.rpm 67 kB/s | 16 kB 00:00 2022-09-20T20:04:42.257 INFO:teuthology.orchestra.run.smithi115.stdout:(54/133): python3-natsort-7.1.1-2.el8.noarch.rp 908 kB/s | 60 kB 00:00 2022-09-20T20:04:42.342 INFO:teuthology.orchestra.run.smithi115.stdout:(55/133): python3-portend-2.6-1.el8.noarch.rpm 185 kB/s | 16 kB 00:00 2022-09-20T20:04:42.361 INFO:teuthology.orchestra.run.smithi036.stdout:(57/133): python3-repoze-lru-0.7-6.el8.noarch.r 255 kB/s | 34 kB 00:00 2022-09-20T20:04:42.374 INFO:teuthology.orchestra.run.smithi115.stdout:(56/133): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2022-09-20T20:04:42.386 INFO:teuthology.orchestra.run.smithi036.stdout:(58/133): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2022-09-20T20:04:42.399 INFO:teuthology.orchestra.run.smithi115.stdout:(57/133): python3-repoze-lru-0.7-6.el8.noarch.r 585 kB/s | 34 kB 00:00 2022-09-20T20:04:42.425 INFO:teuthology.orchestra.run.smithi115.stdout:(58/133): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2022-09-20T20:04:42.428 INFO:teuthology.orchestra.run.smithi036.stdout:(59/133): python3-rsa-4.8-1.el8.noarch.rpm 918 kB/s | 61 kB 00:00 2022-09-20T20:04:42.450 INFO:teuthology.orchestra.run.smithi115.stdout:(59/133): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2022-09-20T20:04:42.454 INFO:teuthology.orchestra.run.smithi036.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 279 kB/s | 19 kB 00:00 2022-09-20T20:04:42.476 INFO:teuthology.orchestra.run.smithi115.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 371 kB/s | 19 kB 00:00 2022-09-20T20:04:42.513 INFO:teuthology.orchestra.run.smithi036.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 286 kB/s | 24 kB 00:00 2022-09-20T20:04:42.517 INFO:teuthology.orchestra.run.smithi115.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 356 kB/s | 24 kB 00:00 2022-09-20T20:04:42.538 INFO:teuthology.orchestra.run.smithi036.stdout:(62/133): python3-tempora-1.14.1-5.el8.noarch.r 350 kB/s | 29 kB 00:00 2022-09-20T20:04:42.543 INFO:teuthology.orchestra.run.smithi115.stdout:(62/133): python3-tempora-1.14.1-5.el8.noarch.r 436 kB/s | 29 kB 00:00 2022-09-20T20:04:42.580 INFO:teuthology.orchestra.run.smithi036.stdout:(63/133): python3-trustme-0.6.0-4.el8.noarch.rp 412 kB/s | 27 kB 00:00 2022-09-20T20:04:42.585 INFO:teuthology.orchestra.run.smithi115.stdout:(63/133): python3-trustme-0.6.0-4.el8.noarch.rp 411 kB/s | 27 kB 00:00 2022-09-20T20:04:42.638 INFO:teuthology.orchestra.run.smithi036.stdout:(64/133): python3-waitress-1.2.1-2.el8.1.noarch 2.3 MB/s | 241 kB 00:00 2022-09-20T20:04:42.662 INFO:teuthology.orchestra.run.smithi115.stdout:(64/133): python3-waitress-1.2.1-2.el8.1.noarch 2.0 MB/s | 241 kB 00:00 2022-09-20T20:04:42.706 INFO:teuthology.orchestra.run.smithi036.stdout:(65/133): python3-websocket-client-0.56.0-5.el8 919 kB/s | 61 kB 00:00 2022-09-20T20:04:42.719 INFO:teuthology.orchestra.run.smithi115.stdout:(65/133): python3-websocket-client-0.56.0-5.el8 1.1 MB/s | 61 kB 00:00 2022-09-20T20:04:42.756 INFO:teuthology.orchestra.run.smithi036.stdout:(66/133): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2022-09-20T20:04:42.789 INFO:teuthology.orchestra.run.smithi036.stdout:(67/133): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2022-09-20T20:04:42.906 INFO:teuthology.orchestra.run.smithi036.stdout:(68/133): python3-zc-lockfile-2.0-2.el8.noarch. 153 kB/s | 23 kB 00:00 2022-09-20T20:04:42.931 INFO:teuthology.orchestra.run.smithi036.stdout:(69/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2022-09-20T20:04:42.982 INFO:teuthology.orchestra.run.smithi036.stdout:(70/133): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2022-09-20T20:04:43.032 INFO:teuthology.orchestra.run.smithi036.stdout:(71/133): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2022-09-20T20:04:43.099 INFO:teuthology.orchestra.run.smithi036.stdout:(72/133): postgresql-10.21-2.module+el8.6.0+153 23 MB/s | 1.5 MB 00:00 2022-09-20T20:04:43.191 INFO:teuthology.orchestra.run.smithi036.stdout:(73/133): python3-numpy-1.14.3-10.el8.x86_64.rp 41 MB/s | 3.7 MB 00:00 2022-09-20T20:04:43.252 INFO:teuthology.orchestra.run.smithi115.stdout:(66/133): ceph-test-17.0.0-15134.gada1013c.el8. 8.0 MB/s | 54 MB 00:06 2022-09-20T20:04:43.278 INFO:teuthology.orchestra.run.smithi115.stdout:(67/133): python3-webob-1.8.5-1.el8.1.noarch.rp 363 kB/s | 251 kB 00:00 2022-09-20T20:04:43.291 INFO:teuthology.orchestra.run.smithi036.stdout:(74/133): postgresql-server-10.21-2.module+el8. 51 MB/s | 5.1 MB 00:00 2022-09-20T20:04:43.303 INFO:teuthology.orchestra.run.smithi115.stdout:(68/133): python3-webtest-2.0.33-1.el8.noarch.r 146 kB/s | 85 kB 00:00 2022-09-20T20:04:43.328 INFO:teuthology.orchestra.run.smithi115.stdout:(69/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2022-09-20T20:04:43.333 INFO:teuthology.orchestra.run.smithi036.stdout:(75/133): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2022-09-20T20:04:43.367 INFO:teuthology.orchestra.run.smithi036.stdout:(76/133): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2022-09-20T20:04:43.378 INFO:teuthology.orchestra.run.smithi115.stdout:(70/133): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2022-09-20T20:04:43.401 INFO:teuthology.orchestra.run.smithi036.stdout:(77/133): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2022-09-20T20:04:43.403 INFO:teuthology.orchestra.run.smithi115.stdout:(71/133): python3-zc-lockfile-2.0-2.el8.noarch. 152 kB/s | 23 kB 00:00 2022-09-20T20:04:43.429 INFO:teuthology.orchestra.run.smithi115.stdout:(72/133): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2022-09-20T20:04:43.434 INFO:teuthology.orchestra.run.smithi036.stdout:(78/133): thrift-0.13.0-2.el8.x86_64.rpm 2.7 MB/s | 1.7 MB 00:00 2022-09-20T20:04:43.454 INFO:teuthology.orchestra.run.smithi115.stdout:(73/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2022-09-20T20:04:43.476 INFO:teuthology.orchestra.run.smithi036.stdout:(79/133): librdkafka-0.11.4-3.el8.x86_64.rpm 4.6 MB/s | 354 kB 00:00 2022-09-20T20:04:43.511 INFO:teuthology.orchestra.run.smithi036.stdout:(80/133): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2022-09-20T20:04:43.538 INFO:teuthology.orchestra.run.smithi115.stdout:(74/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.0 MB/s | 220 kB 00:00 2022-09-20T20:04:43.561 INFO:teuthology.orchestra.run.smithi036.stdout:(81/133): libtool-2.4.6-25.el8.x86_64.rpm 8.3 MB/s | 709 kB 00:00 2022-09-20T20:04:43.563 INFO:teuthology.orchestra.run.smithi115.stdout:(75/133): libselinux-devel-2.9-5.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2022-09-20T20:04:43.586 INFO:teuthology.orchestra.run.smithi036.stdout:(82/133): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.0 MB/s | 225 kB 00:00 2022-09-20T20:04:43.597 INFO:teuthology.orchestra.run.smithi115.stdout:(76/133): thrift-0.13.0-2.el8.x86_64.rpm 5.4 MB/s | 1.7 MB 00:00 2022-09-20T20:04:43.622 INFO:teuthology.orchestra.run.smithi115.stdout:(77/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 213 kB/s | 18 kB 00:00 2022-09-20T20:04:43.627 INFO:teuthology.orchestra.run.smithi036.stdout:(83/133): python3-mako-1.0.6-13.el8.noarch.rpm 3.8 MB/s | 157 kB 00:00 2022-09-20T20:04:43.647 INFO:teuthology.orchestra.run.smithi115.stdout:(78/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 565 kB/s | 47 kB 00:00 2022-09-20T20:04:43.689 INFO:teuthology.orchestra.run.smithi115.stdout:(79/133): python3-cryptography-3.2.1-5.el8.x86_ 6.1 MB/s | 559 kB 00:00 2022-09-20T20:04:43.714 INFO:teuthology.orchestra.run.smithi115.stdout:(80/133): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2022-09-20T20:04:43.719 INFO:teuthology.orchestra.run.smithi036.stdout:(84/133): openblas-threads-0.3.15-3.el8.x86_64. 31 MB/s | 4.9 MB 00:00 2022-09-20T20:04:43.732 INFO:teuthology.orchestra.run.smithi115.stdout:(81/133): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2022-09-20T20:04:43.762 INFO:teuthology.orchestra.run.smithi036.stdout:(85/133): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2022-09-20T20:04:43.773 INFO:teuthology.orchestra.run.smithi115.stdout:(82/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 6.5 MB/s | 560 kB 00:00 2022-09-20T20:04:43.821 INFO:teuthology.orchestra.run.smithi115.stdout:(83/133): python3-jwt-1.6.1-2.el8.noarch.rpm 477 kB/s | 43 kB 00:00 2022-09-20T20:04:43.849 INFO:teuthology.orchestra.run.smithi115.stdout:(84/133): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.3 MB/s | 171 kB 00:00 2022-09-20T20:04:43.859 INFO:teuthology.orchestra.run.smithi036.stdout:(86/133): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2022-09-20T20:04:43.874 INFO:teuthology.orchestra.run.smithi115.stdout:(85/133): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 574 kB/s | 58 kB 00:00 2022-09-20T20:04:43.888 INFO:teuthology.orchestra.run.smithi036.stdout:(87/133): socat-1.7.4.1-1.el8.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2022-09-20T20:04:43.891 INFO:teuthology.orchestra.run.smithi115.stdout:(86/133): elfutils-libelf-devel-0.186-1.el8.x86 862 kB/s | 60 kB 00:00 2022-09-20T20:04:43.917 INFO:teuthology.orchestra.run.smithi115.stdout:(87/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2022-09-20T20:04:43.921 INFO:teuthology.orchestra.run.smithi036.stdout:(88/133): flex-2.6.1-9.el8.x86_64.rpm 5.0 MB/s | 320 kB 00:00 2022-09-20T20:04:43.942 INFO:teuthology.orchestra.run.smithi115.stdout:(88/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 278 kB/s | 19 kB 00:00 2022-09-20T20:04:43.963 INFO:teuthology.orchestra.run.smithi036.stdout:(89/133): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2022-09-20T20:04:43.967 INFO:teuthology.orchestra.run.smithi115.stdout:(89/133): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.3 MB/s | 98 kB 00:00 2022-09-20T20:04:43.993 INFO:teuthology.orchestra.run.smithi115.stdout:(90/133): NetworkManager-initscripts-updown-1.3 1.8 MB/s | 137 kB 00:00 2022-09-20T20:04:44.026 INFO:teuthology.orchestra.run.smithi115.stdout:(91/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2022-09-20T20:04:44.039 INFO:teuthology.orchestra.run.smithi036.stdout:(90/133): openblas-0.3.15-3.el8.x86_64.rpm 31 MB/s | 4.7 MB 00:00 2022-09-20T20:04:44.052 INFO:teuthology.orchestra.run.smithi115.stdout:(92/133): python3-requests-oauthlib-1.0.0-1.el8 517 kB/s | 43 kB 00:00 2022-09-20T20:04:44.077 INFO:teuthology.orchestra.run.smithi115.stdout:(93/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 863 kB/s | 72 kB 00:00 2022-09-20T20:04:44.080 INFO:teuthology.orchestra.run.smithi036.stdout:(91/133): automake-1.16.1-7.el8.noarch.rpm 17 MB/s | 713 kB 00:00 2022-09-20T20:04:44.114 INFO:teuthology.orchestra.run.smithi036.stdout:(92/133): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2022-09-20T20:04:44.139 INFO:teuthology.orchestra.run.smithi036.stdout:(93/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 4.9 MB/s | 126 kB 00:00 2022-09-20T20:04:44.141 INFO:teuthology.orchestra.run.smithi115.stdout:(94/133): libcom_err-devel-1.45.6-4.el8.x86_64. 337 kB/s | 39 kB 00:00 2022-09-20T20:04:44.169 INFO:teuthology.orchestra.run.smithi115.stdout:(95/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 529 kB/s | 48 kB 00:00 2022-09-20T20:04:44.173 INFO:teuthology.orchestra.run.smithi036.stdout:(94/133): python3-jsonpointer-1.10-11.el8.noarc 599 kB/s | 20 kB 00:00 2022-09-20T20:04:44.215 INFO:teuthology.orchestra.run.smithi036.stdout:(95/133): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2022-09-20T20:04:44.227 INFO:teuthology.orchestra.run.smithi115.stdout:(96/133): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2022-09-20T20:04:44.257 INFO:teuthology.orchestra.run.smithi036.stdout:(96/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2022-09-20T20:04:44.261 INFO:teuthology.orchestra.run.smithi115.stdout:(97/133): libgfortran-8.5.0-10.1.el8_6.x86_64.r 5.3 MB/s | 644 kB 00:00 2022-09-20T20:04:44.295 INFO:teuthology.orchestra.run.smithi115.stdout:(98/133): postgresql-contrib-10.21-2.module+el8 6.3 MB/s | 808 kB 00:00 2022-09-20T20:04:44.337 INFO:teuthology.orchestra.run.smithi115.stdout:(99/133): bison-3.0.4-10.el8.x86_64.rpm 6.2 MB/s | 688 kB 00:00 2022-09-20T20:04:44.387 INFO:teuthology.orchestra.run.smithi115.stdout:(100/133): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2022-09-20T20:04:44.390 INFO:teuthology.orchestra.run.smithi036.stdout:(97/133): python3-scipy-1.0.0-21.module+el8.5.0 32 MB/s | 14 MB 00:00 2022-09-20T20:04:44.429 INFO:teuthology.orchestra.run.smithi115.stdout:(101/133): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2022-09-20T20:04:44.432 INFO:teuthology.orchestra.run.smithi036.stdout:(98/133): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2022-09-20T20:04:44.458 INFO:teuthology.orchestra.run.smithi036.stdout:(99/133): python3-jsonpatch-1.21-2.el8.noarch.r 398 kB/s | 27 kB 00:00 2022-09-20T20:04:44.491 INFO:teuthology.orchestra.run.smithi036.stdout:(100/133): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2022-09-20T20:04:44.504 INFO:teuthology.orchestra.run.smithi115.stdout:(102/133): python3-numpy-1.14.3-10.el8.x86_64.r 18 MB/s | 3.7 MB 00:00 2022-09-20T20:04:44.517 INFO:teuthology.orchestra.run.smithi036.stdout:(101/133): python3-pyasn1-modules-0.3.7-6.el8.n 1.8 MB/s | 110 kB 00:00 2022-09-20T20:04:44.538 INFO:teuthology.orchestra.run.smithi115.stdout:(103/133): python3-markupsafe-0.23-19.el8.x86_6 358 kB/s | 39 kB 00:00 2022-09-20T20:04:44.559 INFO:teuthology.orchestra.run.smithi036.stdout:(102/133): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2022-09-20T20:04:44.563 INFO:teuthology.orchestra.run.smithi115.stdout:(104/133): python3-prettytable-0.7.2-14.el8.noa 767 kB/s | 44 kB 00:00 2022-09-20T20:04:44.575 INFO:teuthology.orchestra.run.smithi036.stdout:(103/133): fuse-2.9.7-15.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2022-09-20T20:04:44.609 INFO:teuthology.orchestra.run.smithi036.stdout:(104/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2022-09-20T20:04:44.635 INFO:teuthology.orchestra.run.smithi036.stdout:(105/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2022-09-20T20:04:44.646 INFO:teuthology.orchestra.run.smithi115.stdout:(105/133): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2022-09-20T20:04:44.668 INFO:teuthology.orchestra.run.smithi036.stdout:(106/133): libselinux-devel-2.9-5.el8.x86_64.rp 3.3 MB/s | 200 kB 00:00 2022-09-20T20:04:44.672 INFO:teuthology.orchestra.run.smithi115.stdout:(106/133): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2022-09-20T20:04:44.693 INFO:teuthology.orchestra.run.smithi036.stdout:(107/133): libverto-devel-0.3.0-5.el8.x86_64.rp 311 kB/s | 18 kB 00:00 2022-09-20T20:04:44.697 INFO:teuthology.orchestra.run.smithi115.stdout:(107/133): python3-jinja2-2.10.1-3.el8.noarch.r 3.9 MB/s | 538 kB 00:00 2022-09-20T20:04:44.727 INFO:teuthology.orchestra.run.smithi036.stdout:(108/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 797 kB/s | 47 kB 00:00 2022-09-20T20:04:44.739 INFO:teuthology.orchestra.run.smithi115.stdout:(108/133): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2022-09-20T20:04:44.769 INFO:teuthology.orchestra.run.smithi036.stdout:(109/133): python3-cryptography-3.2.1-5.el8.x86 7.3 MB/s | 559 kB 00:00 2022-09-20T20:04:44.789 INFO:teuthology.orchestra.run.smithi115.stdout:(109/133): python3-numpy-f2py-1.14.3-10.el8.x86 1.9 MB/s | 225 kB 00:00 2022-09-20T20:04:44.820 INFO:teuthology.orchestra.run.smithi036.stdout:(110/133): python3-oauthlib-2.1.0-1.el8.noarch. 1.6 MB/s | 155 kB 00:00 2022-09-20T20:04:44.831 INFO:teuthology.orchestra.run.smithi115.stdout:(110/133): python3-mako-1.0.6-13.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2022-09-20T20:04:44.853 INFO:teuthology.orchestra.run.smithi036.stdout:(111/133): python3-pycparser-2.14-14.el8.noarch 1.3 MB/s | 109 kB 00:00 2022-09-20T20:04:44.873 INFO:teuthology.orchestra.run.smithi115.stdout:(111/133): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2022-09-20T20:04:44.895 INFO:teuthology.orchestra.run.smithi036.stdout:(112/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2022-09-20T20:04:44.929 INFO:teuthology.orchestra.run.smithi036.stdout:(113/133): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2022-09-20T20:04:44.954 INFO:teuthology.orchestra.run.smithi036.stdout:(114/133): python3-jwt-1.6.1-2.el8.noarch.rpm 724 kB/s | 43 kB 00:00 2022-09-20T20:04:44.957 INFO:teuthology.orchestra.run.smithi115.stdout:(112/133): openblas-threads-0.3.15-3.el8.x86_64 19 MB/s | 4.9 MB 00:00 2022-09-20T20:04:44.971 INFO:teuthology.orchestra.run.smithi036.stdout:(115/133): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.4 MB/s | 58 kB 00:00 2022-09-20T20:04:44.990 INFO:teuthology.orchestra.run.smithi115.stdout:(113/133): socat-1.7.4.1-1.el8.x86_64.rpm 2.7 MB/s | 323 kB 00:00 2022-09-20T20:04:44.996 INFO:teuthology.orchestra.run.smithi036.stdout:(116/133): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2022-09-20T20:04:45.013 INFO:teuthology.orchestra.run.smithi036.stdout:(117/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2022-09-20T20:04:45.047 INFO:teuthology.orchestra.run.smithi036.stdout:(118/133): libaio-devel-0.3.112-1.el8.x86_64.rp 371 kB/s | 19 kB 00:00 2022-09-20T20:04:45.066 INFO:teuthology.orchestra.run.smithi115.stdout:(114/133): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2022-09-20T20:04:45.072 INFO:teuthology.orchestra.run.smithi036.stdout:(119/133): libuuid-devel-2.32.1-35.el8.x86_64.r 1.7 MB/s | 98 kB 00:00 2022-09-20T20:04:45.114 INFO:teuthology.orchestra.run.smithi036.stdout:(120/133): NetworkManager-initscripts-updown-1. 2.0 MB/s | 137 kB 00:00 2022-09-20T20:04:45.133 INFO:teuthology.orchestra.run.smithi115.stdout:(115/133): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2022-09-20T20:04:45.140 INFO:teuthology.orchestra.run.smithi036.stdout:(121/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.4 MB/s | 229 kB 00:00 2022-09-20T20:04:45.166 INFO:teuthology.orchestra.run.smithi115.stdout:(116/133): jq-1.6-3.el8.x86_64.rpm 2.0 MB/s | 202 kB 00:00 2022-09-20T20:04:45.173 INFO:teuthology.orchestra.run.smithi036.stdout:(122/133): python3-requests-oauthlib-1.0.0-1.el 727 kB/s | 43 kB 00:00 2022-09-20T20:04:45.199 INFO:teuthology.orchestra.run.smithi036.stdout:(123/133): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2022-09-20T20:04:45.216 INFO:teuthology.orchestra.run.smithi115.stdout:(117/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-09-20T20:04:45.232 INFO:teuthology.orchestra.run.smithi036.stdout:(124/133): libcom_err-devel-1.45.6-4.el8.x86_64 659 kB/s | 39 kB 00:00 2022-09-20T20:04:45.267 INFO:teuthology.orchestra.run.smithi036.stdout:(125/133): keyutils-libs-devel-1.5.10-9.el8.x86 1.4 MB/s | 48 kB 00:00 2022-09-20T20:04:45.292 INFO:teuthology.orchestra.run.smithi115.stdout:(118/133): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2022-09-20T20:04:45.317 INFO:teuthology.orchestra.run.smithi036.stdout:(126/133): openssl-devel-1.1.1k-7.el8_6.x86_64. 20 MB/s | 2.3 MB 00:00 2022-09-20T20:04:45.317 INFO:teuthology.orchestra.run.smithi115.stdout:(119/133): python3-pytz-2017.2-9.el8.noarch.rpm 541 kB/s | 54 kB 00:00 2022-09-20T20:04:45.351 INFO:teuthology.orchestra.run.smithi115.stdout:(120/133): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.1 MB/s | 126 kB 00:00 2022-09-20T20:04:45.366 INFO:teuthology.orchestra.run.smithi036.stdout:(127/133): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.3 MB/s | 644 kB 00:00 2022-09-20T20:04:45.367 INFO:teuthology.orchestra.run.smithi115.stdout:(121/133): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2022-09-20T20:04:45.420 INFO:teuthology.orchestra.run.smithi115.stdout:(122/133): libpq-13.5-1.el8.x86_64.rpm 2.8 MB/s | 198 kB 00:00 2022-09-20T20:04:45.452 INFO:teuthology.orchestra.run.smithi115.stdout:(123/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2022-09-20T20:04:45.479 INFO:teuthology.orchestra.run.smithi115.stdout:(124/133): oniguruma-6.8.2-2.el8.x86_64.rpm 3.3 MB/s | 187 kB 00:00 2022-09-20T20:04:45.502 INFO:teuthology.orchestra.run.smithi115.stdout:(125/133): python3-jsonpatch-1.21-2.el8.noarch. 541 kB/s | 27 kB 00:00 2022-09-20T20:04:45.536 INFO:teuthology.orchestra.run.smithi115.stdout:(126/133): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2022-09-20T20:04:45.561 INFO:teuthology.orchestra.run.smithi115.stdout:(127/133): python3-pyasn1-modules-0.3.7-6.el8.n 1.8 MB/s | 110 kB 00:00 2022-09-20T20:04:45.712 INFO:teuthology.orchestra.run.smithi115.stdout:(128/133): python3-scipy-1.0.0-21.module+el8.5. 24 MB/s | 14 MB 00:00 2022-09-20T20:04:45.778 INFO:teuthology.orchestra.run.smithi115.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 10 MB/s | 709 kB 00:00 2022-09-20T20:04:45.828 INFO:teuthology.orchestra.run.smithi115.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 29 MB/s | 1.5 MB 00:00 2022-09-20T20:04:45.858 INFO:teuthology.orchestra.run.smithi036.stdout:(128/133): ceph-test-17.0.0-15134.gada1013c.el8 5.2 MB/s | 54 MB 00:10 2022-09-20T20:04:45.870 INFO:teuthology.orchestra.run.smithi115.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2022-09-20T20:04:45.900 INFO:teuthology.orchestra.run.smithi036.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 17 MB/s | 709 kB 00:00 2022-09-20T20:04:45.967 INFO:teuthology.orchestra.run.smithi036.stdout:(130/133): librados2-17.0.0-15134.gada1013c.el8 5.6 MB/s | 3.6 MB 00:00 2022-09-20T20:04:46.018 INFO:teuthology.orchestra.run.smithi036.stdout:(131/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 1.5 MB 00:00 2022-09-20T20:04:46.051 INFO:teuthology.orchestra.run.smithi036.stdout:(132/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 9.4 MB/s | 798 kB 00:00 2022-09-20T20:04:46.112 INFO:teuthology.orchestra.run.smithi115.stdout:(132/133): librbd1-17.0.0-15134.gada1013c.el8.x 6.5 MB/s | 3.6 MB 00:00 2022-09-20T20:04:46.296 INFO:teuthology.orchestra.run.smithi115.stdout:(133/133): librados2-17.0.0-15134.gada1013c.el8 4.8 MB/s | 3.6 MB 00:00 2022-09-20T20:04:46.298 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2022-09-20T20:04:46.299 INFO:teuthology.orchestra.run.smithi115.stdout:Total 13 MB/s | 218 MB 00:16 2022-09-20T20:04:46.385 INFO:teuthology.orchestra.run.smithi036.stdout:(133/133): librbd1-17.0.0-15134.gada1013c.el8.x 3.5 MB/s | 3.6 MB 00:01 2022-09-20T20:04:46.387 INFO:teuthology.orchestra.run.smithi036.stdout:-------------------------------------------------------------------------------- 2022-09-20T20:04:46.388 INFO:teuthology.orchestra.run.smithi036.stdout:Total 14 MB/s | 218 MB 00:15 2022-09-20T20:04:47.149 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2022-09-20T20:04:47.297 INFO:teuthology.orchestra.run.smithi036.stdout:Running transaction check 2022-09-20T20:04:47.307 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2022-09-20T20:04:47.308 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2022-09-20T20:04:47.474 INFO:teuthology.orchestra.run.smithi036.stdout:Transaction check succeeded. 2022-09-20T20:04:47.474 INFO:teuthology.orchestra.run.smithi036.stdout:Running transaction test 2022-09-20T20:04:48.466 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2022-09-20T20:04:48.470 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2022-09-20T20:04:48.704 INFO:teuthology.orchestra.run.smithi036.stdout:Transaction test succeeded. 2022-09-20T20:04:48.709 INFO:teuthology.orchestra.run.smithi036.stdout:Running transaction 2022-09-20T20:04:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2022-09-20T20:04:53.650 INFO:teuthology.orchestra.run.smithi036.stdout: Preparing : 1/1 2022-09-20T20:04:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-09-20T20:04:53.847 INFO:teuthology.orchestra.run.smithi036.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-09-20T20:04:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-09-20T20:04:54.067 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-09-20T20:04:54.215 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-09-20T20:04:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-09-20T20:04:54.372 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-09-20T20:04:54.384 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-09-20T20:04:54.541 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/138 2022-09-20T20:04:54.551 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/138 2022-09-20T20:04:54.621 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/138 2022-09-20T20:04:54.702 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/138 2022-09-20T20:04:54.802 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/138 2022-09-20T20:04:54.830 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 5/138 2022-09-20T20:04:54.985 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-09-20T20:04:54.992 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-09-20T20:04:55.377 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-09-20T20:04:55.415 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-09-20T20:04:55.626 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:17.0.0-15134.gada1013c.el8.x86_64 7/138 2022-09-20T20:04:55.659 INFO:teuthology.orchestra.run.smithi036.stdout: Upgrading : librados2-2:17.0.0-15134.gada1013c.el8.x86_64 7/138 2022-09-20T20:04:56.017 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:17.0.0-15134.gada1013c.el8.x86_64 7/138 2022-09-20T20:04:56.071 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: librados2-2:17.0.0-15134.gada1013c.el8.x86_64 7/138 2022-09-20T20:04:56.226 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:17.0.0-15134.gada1013c.el8.x86_64 8/138 2022-09-20T20:04:56.242 INFO:teuthology.orchestra.run.smithi036.stdout: Upgrading : librbd1-2:17.0.0-15134.gada1013c.el8.x86_64 8/138 2022-09-20T20:04:56.354 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:17.0.0-15134.gada1013c.el8.x86_64 8/138 2022-09-20T20:04:56.356 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: librbd1-2:17.0.0-15134.gada1013c.el8.x86_64 8/138 2022-09-20T20:04:56.567 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libcephfs2-2:17.0.0-15134.gada1013c.el8.x86_64 9/138 2022-09-20T20:04:56.584 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:17.0.0-15134.gada1013c.el8.x86_64 9/138 2022-09-20T20:04:56.652 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: libcephfs2-2:17.0.0-15134.gada1013c.el8.x86_64 9/138 2022-09-20T20:04:56.668 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:17.0.0-15134.gada1013c.el8.x86_64 9/138 2022-09-20T20:04:56.808 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-rados-2:17.0.0-15134.gada1013c.el8.x86_6 10/138 2022-09-20T20:04:56.899 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:17.0.0-15134.gada1013c.el8.x86_6 10/138 2022-09-20T20:04:57.008 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-09-20T20:04:57.154 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-09-20T20:04:57.187 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-09-20T20:04:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-09-20T20:04:57.530 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-09-20T20:04:57.722 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-09-20T20:04:57.760 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-09-20T20:04:57.875 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libcephsqlite-2:17.0.0-15134.gada1013c.el8.x86_6 15/138 2022-09-20T20:04:58.005 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-09-20T20:04:58.125 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: libcephsqlite-2:17.0.0-15134.gada1013c.el8.x86_6 15/138 2022-09-20T20:04:58.201 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:17.0.0-15134.gada1013c.el8.x86_6 15/138 2022-09-20T20:04:58.307 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:17.0.0-15134.gada1013c.el8.x86_6 15/138 2022-09-20T20:04:58.567 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libradosstriper1-2:17.0.0-15134.gada1013c.el8.x8 16/138 2022-09-20T20:04:58.621 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: libradosstriper1-2:17.0.0-15134.gada1013c.el8.x8 16/138 2022-09-20T20:04:58.751 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:17.0.0-15134.gada1013c.el8.x8 16/138 2022-09-20T20:04:58.774 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 17/138 2022-09-20T20:04:58.830 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:17.0.0-15134.gada1013c.el8.x8 16/138 2022-09-20T20:04:58.892 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-09-20T20:04:59.006 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 17/138 2022-09-20T20:04:59.016 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-09-20T20:04:59.134 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-09-20T20:04:59.232 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/138 2022-09-20T20:04:59.415 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/138 2022-09-20T20:04:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 20/138 2022-09-20T20:04:59.734 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-09-20T20:04:59.776 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-09-20T20:04:59.876 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-09-20T20:04:59.925 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-09-20T20:05:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-09-20T20:05:00.539 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-09-20T20:05:00.632 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-09-20T20:05:00.701 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : openblas-0.3.15-3.el8.x86_64 21/138 2022-09-20T20:05:00.850 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-09-20T20:05:01.428 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 21/138 2022-09-20T20:05:01.431 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-09-20T20:05:01.617 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.15-3.el8.x86_64 24/138 2022-09-20T20:05:01.699 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 22/138 2022-09-20T20:05:01.846 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 23/138 2022-09-20T20:05:02.115 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/138 2022-09-20T20:05:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 24/138 2022-09-20T20:05:02.401 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/138 2022-09-20T20:05:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/138 2022-09-20T20:05:02.661 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-09-20T20:05:02.831 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-ceph-common-2:17.0.0-15134.gada1013c.el8 27/138 2022-09-20T20:05:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-09-20T20:05:02.953 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-ceph-argparse-2:17.0.0-15134.gada1013c.e 28/138 2022-09-20T20:05:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:17.0.0-15134.gada1013c.el8 27/138 2022-09-20T20:05:03.179 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-cephfs-2:17.0.0-15134.gada1013c.el8.x86_ 29/138 2022-09-20T20:05:03.235 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:17.0.0-15134.gada1013c.e 28/138 2022-09-20T20:05:03.519 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:17.0.0-15134.gada1013c.el8.x86_ 29/138 2022-09-20T20:05:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-09-20T20:05:04.127 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 30/138 2022-09-20T20:05:04.471 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/138 2022-09-20T20:05:04.741 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/138 2022-09-20T20:05:04.950 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-09-20T20:05:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-09-20T20:05:05.867 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/138 2022-09-20T20:05:06.808 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-09-20T20:05:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/138 2022-09-20T20:05:07.145 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/138 2022-09-20T20:05:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/138 2022-09-20T20:05:07.390 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-09-20T20:05:07.510 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/138 2022-09-20T20:05:07.595 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-09-20T20:05:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-09-20T20:05:07.814 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-rbd-2:17.0.0-15134.gada1013c.el8.x86_64 37/138 2022-09-20T20:05:07.925 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-09-20T20:05:08.058 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : librados-devel-2:17.0.0-15134.gada1013c.el8.x86_ 38/138 2022-09-20T20:05:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:17.0.0-15134.gada1013c.el8.x86_64 37/138 2022-09-20T20:05:08.256 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-09-20T20:05:08.379 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 40/138 2022-09-20T20:05:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:17.0.0-15134.gada1013c.el8.x86_ 38/138 2022-09-20T20:05:08.538 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 41/138 2022-09-20T20:05:08.616 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-09-20T20:05:08.724 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/138 2022-09-20T20:05:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/138 2022-09-20T20:05:08.917 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 43/138 2022-09-20T20:05:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/138 2022-09-20T20:05:09.147 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 41/138 2022-09-20T20:05:09.166 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 44/138 2022-09-20T20:05:09.225 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 41/138 2022-09-20T20:05:09.473 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : jq-1.6-3.el8.x86_64 42/138 2022-09-20T20:05:09.584 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 45/138 2022-09-20T20:05:09.631 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 43/138 2022-09-20T20:05:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 44/138 2022-09-20T20:05:09.922 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 46/138 2022-09-20T20:05:10.160 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 47/138 2022-09-20T20:05:10.181 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : automake-1.16.1-7.el8.noarch 45/138 2022-09-20T20:05:10.377 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 46/138 2022-09-20T20:05:10.412 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 48/138 2022-09-20T20:05:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 47/138 2022-09-20T20:05:10.658 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 49/138 2022-09-20T20:05:10.702 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/138 2022-09-20T20:05:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 49/138 2022-09-20T20:05:11.024 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 50/138 2022-09-20T20:05:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 50/138 2022-09-20T20:05:11.224 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 51/138 2022-09-20T20:05:11.400 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 52/138 2022-09-20T20:05:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/138 2022-09-20T20:05:11.618 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 53/138 2022-09-20T20:05:11.762 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 54/138 2022-09-20T20:05:11.801 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/138 2022-09-20T20:05:12.103 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/138 2022-09-20T20:05:12.132 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/138 2022-09-20T20:05:12.279 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/138 2022-09-20T20:05:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/138 2022-09-20T20:05:12.502 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 57/138 2022-09-20T20:05:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/138 2022-09-20T20:05:12.739 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 58/138 2022-09-20T20:05:12.801 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/138 2022-09-20T20:05:13.008 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 59/138 2022-09-20T20:05:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/138 2022-09-20T20:05:13.216 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : fuse-2.9.7-15.el8.x86_64 60/138 2022-09-20T20:05:13.390 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/138 2022-09-20T20:05:13.405 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/138 2022-09-20T20:05:13.462 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/138 2022-09-20T20:05:13.590 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/138 2022-09-20T20:05:13.614 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 62/138 2022-09-20T20:05:13.693 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 62/138 2022-09-20T20:05:13.833 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/138 2022-09-20T20:05:13.890 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : jq-1.6-3.el8.x86_64 63/138 2022-09-20T20:05:14.009 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 61/138 2022-09-20T20:05:14.057 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 64/138 2022-09-20T20:05:14.323 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 65/138 2022-09-20T20:05:14.558 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : automake-1.16.1-7.el8.noarch 66/138 2022-09-20T20:05:14.682 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/138 2022-09-20T20:05:14.760 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 67/138 2022-09-20T20:05:14.884 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/138 2022-09-20T20:05:14.941 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-09-20T20:05:15.139 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-09-20T20:05:15.175 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 64/138 2022-09-20T20:05:15.405 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-09-20T20:05:15.408 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/138 2022-09-20T20:05:15.589 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-09-20T20:05:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/138 2022-09-20T20:05:15.778 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-09-20T20:05:15.855 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.7-15.el8.x86_64 67/138 2022-09-20T20:05:15.935 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-09-20T20:05:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-09-20T20:05:16.127 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-09-20T20:05:16.224 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-09-20T20:05:16.329 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-09-20T20:05:16.509 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-09-20T20:05:16.520 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-09-20T20:05:16.726 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-09-20T20:05:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-09-20T20:05:16.881 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-09-20T20:05:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-09-20T20:05:17.090 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-09-20T20:05:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-09-20T20:05:17.302 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-09-20T20:05:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-09-20T20:05:17.471 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-09-20T20:05:17.515 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-09-20T20:05:17.715 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-09-20T20:05:17.750 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-09-20T20:05:17.874 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-09-20T20:05:17.931 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-09-20T20:05:18.060 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-09-20T20:05:18.116 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-09-20T20:05:18.290 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-09-20T20:05:18.342 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-09-20T20:05:18.499 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-09-20T20:05:18.711 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-09-20T20:05:18.990 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-09-20T20:05:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-09-20T20:05:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-09-20T20:05:19.508 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-09-20T20:05:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-09-20T20:05:19.762 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-09-20T20:05:19.973 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-09-20T20:05:20.281 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-09-20T20:05:20.565 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-09-20T20:05:20.766 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-09-20T20:05:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-09-20T20:05:21.098 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-09-20T20:05:21.345 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-09-20T20:05:21.566 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-09-20T20:05:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-09-20T20:05:21.731 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : librgw2-2:17.0.0-15134.gada1013c.el8.x86_64 93/138 2022-09-20T20:05:21.791 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: librgw2-2:17.0.0-15134.gada1013c.el8.x86_64 93/138 2022-09-20T20:05:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-09-20T20:05:21.981 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : python3-rgw-2:17.0.0-15134.gada1013c.el8.x86_64 94/138 2022-09-20T20:05:22.136 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-09-20T20:05:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-09-20T20:05:23.045 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:17.0.0-15134.gada1013c.el8.x86_64 93/138 2022-09-20T20:05:23.141 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:17.0.0-15134.gada1013c.el8.x86_64 93/138 2022-09-20T20:05:23.321 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:17.0.0-15134.gada1013c.el8.x86_64 94/138 2022-09-20T20:05:25.991 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 95/138 2022-09-20T20:05:26.180 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 95/138 2022-09-20T20:05:26.770 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 95/138 2022-09-20T20:05:27.010 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-base-2:17.0.0-15134.gada1013c.el8.x86_64 96/138 2022-09-20T20:05:27.079 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-base-2:17.0.0-15134.gada1013c.el8.x86_64 96/138 2022-09-20T20:05:27.108 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 95/138 2022-09-20T20:05:27.221 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-selinux-2:17.0.0-15134.gada1013c.el8.x86_64 97/138 2022-09-20T20:05:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 95/138 2022-09-20T20:05:27.897 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 95/138 2022-09-20T20:05:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:17.0.0-15134.gada1013c.el8.x86_64 96/138 2022-09-20T20:05:28.275 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:17.0.0-15134.gada1013c.el8.x86_64 96/138 2022-09-20T20:05:28.428 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:17.0.0-15134.gada1013c.el8.x86_64 97/138 2022-09-20T20:05:46.526 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-selinux-2:17.0.0-15134.gada1013c.el8.x86_64 97/138 2022-09-20T20:05:46.527 INFO:teuthology.orchestra.run.smithi036.stdout:skipping the directory /sys 2022-09-20T20:05:46.527 INFO:teuthology.orchestra.run.smithi036.stdout:skipping the directory /proc 2022-09-20T20:05:46.527 INFO:teuthology.orchestra.run.smithi036.stdout:skipping the directory /dev 2022-09-20T20:05:46.527 INFO:teuthology.orchestra.run.smithi036.stdout:skipping the directory /run 2022-09-20T20:05:46.528 INFO:teuthology.orchestra.run.smithi036.stdout:skipping the directory /mnt 2022-09-20T20:05:46.528 INFO:teuthology.orchestra.run.smithi036.stdout:skipping the directory /var/tmp 2022-09-20T20:05:46.528 INFO:teuthology.orchestra.run.smithi036.stdout:skipping the directory /home 2022-09-20T20:05:46.529 INFO:teuthology.orchestra.run.smithi036.stdout:skipping the directory /tmp 2022-09-20T20:05:46.529 INFO:teuthology.orchestra.run.smithi036.stdout:skipping the directory /dev 2022-09-20T20:05:46.529 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:05:46.886 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-volume-2:17.0.0-15134.gada1013c.el8.noarch 98/138 2022-09-20T20:05:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:17.0.0-15134.gada1013c.el8.x86_64 97/138 2022-09-20T20:05:47.323 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2022-09-20T20:05:47.323 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2022-09-20T20:05:47.323 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /dev 2022-09-20T20:05:47.323 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /run 2022-09-20T20:05:47.324 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2022-09-20T20:05:47.324 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2022-09-20T20:05:47.324 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2022-09-20T20:05:47.324 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2022-09-20T20:05:47.324 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /dev 2022-09-20T20:05:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:05:47.698 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:17.0.0-15134.gada1013c.el8.noarch 98/138 2022-09-20T20:05:48.465 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-volume-2:17.0.0-15134.gada1013c.el8.noarch 98/138 2022-09-20T20:05:48.640 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-osd-2:17.0.0-15134.gada1013c.el8.x86_64 99/138 2022-09-20T20:05:48.870 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-osd-2:17.0.0-15134.gada1013c.el8.x86_64 99/138 2022-09-20T20:05:49.002 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-mds-2:17.0.0-15134.gada1013c.el8.x86_64 100/138 2022-09-20T20:05:49.182 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:17.0.0-15134.gada1013c.el8.noarch 98/138 2022-09-20T20:05:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:17.0.0-15134.gada1013c.el8.x86_64 99/138 2022-09-20T20:05:49.406 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-mds-2:17.0.0-15134.gada1013c.el8.x86_64 100/138 2022-09-20T20:05:49.577 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-mon-2:17.0.0-15134.gada1013c.el8.x86_64 101/138 2022-09-20T20:05:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:17.0.0-15134.gada1013c.el8.x86_64 99/138 2022-09-20T20:05:49.595 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-mon-2:17.0.0-15134.gada1013c.el8.x86_64 101/138 2022-09-20T20:05:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:17.0.0-15134.gada1013c.el8.x86_64 100/138 2022-09-20T20:05:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:17.0.0-15134.gada1013c.el8.x86_64 100/138 2022-09-20T20:05:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:17.0.0-15134.gada1013c.el8.x86_64 101/138 2022-09-20T20:05:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:17.0.0-15134.gada1013c.el8.x86_64 101/138 2022-09-20T20:05:51.283 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: cephadm-2:17.0.0-15134.gada1013c.el8.noarch 102/138 2022-09-20T20:05:51.485 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : cephadm-2:17.0.0-15134.gada1013c.el8.noarch 102/138 2022-09-20T20:05:51.609 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-prometheus-alerts-2:17.0.0-15134.gada1013c. 103/138 2022-09-20T20:05:51.797 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-grafana-dashboards-2:17.0.0-15134.gada1013c 104/138 2022-09-20T20:05:51.943 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-mgr-cephadm-2:17.0.0-15134.gada1013c.el8.no 105/138 2022-09-20T20:05:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:17.0.0-15134.gada1013c.el8.noarch 102/138 2022-09-20T20:05:52.156 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.0.0-15134.gada1013c.el8.no 105/138 2022-09-20T20:05:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:17.0.0-15134.gada1013c.el8.noarch 102/138 2022-09-20T20:05:52.345 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:17.0.0-15134.gada1013c. 103/138 2022-09-20T20:05:52.431 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-mgr-dashboard-2:17.0.0-15134.gada1013c.el8. 106/138 2022-09-20T20:05:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:17.0.0-15134.gada1013c 104/138 2022-09-20T20:05:52.730 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:17.0.0-15134.gada1013c.el8.no 105/138 2022-09-20T20:05:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.0.0-15134.gada1013c.el8.no 105/138 2022-09-20T20:05:53.165 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:17.0.0-15134.gada1013c.el8. 106/138 2022-09-20T20:05:53.456 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.0.0-15134.gada1013c.el8. 106/138 2022-09-20T20:05:53.701 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-mgr-diskprediction-local-2:17.0.0-15134.gad 107/138 2022-09-20T20:05:53.798 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.0.0-15134.gad 107/138 2022-09-20T20:05:54.121 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.0.0-15134.gada1013c.el8. 106/138 2022-09-20T20:05:54.267 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-mgr-modules-core-2:17.0.0-15134.gada1013c.e 108/138 2022-09-20T20:05:54.321 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:17.0.0-15134.gad 107/138 2022-09-20T20:05:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.0.0-15134.gad 107/138 2022-09-20T20:05:54.446 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-mgr-2:17.0.0-15134.gada1013c.el8.x86_64 109/138 2022-09-20T20:05:54.483 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-mgr-2:17.0.0-15134.gada1013c.el8.x86_64 109/138 2022-09-20T20:05:54.651 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-mgr-rook-2:17.0.0-15134.gada1013c.el8.noarc 110/138 2022-09-20T20:05:54.663 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-mgr-rook-2:17.0.0-15134.gada1013c.el8.noarc 110/138 2022-09-20T20:05:54.807 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:17.0.0-15134.gada1013c.e 108/138 2022-09-20T20:05:55.034 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:17.0.0-15134.gada1013c.el8.x86_64 109/138 2022-09-20T20:05:55.069 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:17.0.0-15134.gada1013c.el8.x86_64 109/138 2022-09-20T20:05:55.288 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:17.0.0-15134.gada1013c.el8.noarc 110/138 2022-09-20T20:05:55.299 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:17.0.0-15134.gada1013c.el8.noarc 110/138 2022-09-20T20:05:56.423 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-2:17.0.0-15134.gada1013c.el8.x86_64 111/138 2022-09-20T20:05:56.609 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-radosgw-2:17.0.0-15134.gada1013c.el8.x86_64 112/138 2022-09-20T20:05:56.645 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-radosgw-2:17.0.0-15134.gada1013c.el8.x86_64 112/138 2022-09-20T20:05:56.793 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-immutable-object-cache-2:17.0.0-15134.gada1 113/138 2022-09-20T20:05:56.837 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.0.0-15134.gada1 113/138 2022-09-20T20:05:56.946 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:17.0.0-15134.gada1013c.el8.x86_64 111/138 2022-09-20T20:05:56.993 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : cephfs-mirror-2:17.0.0-15134.gada1013c.el8.x86_6 114/138 2022-09-20T20:05:57.137 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:17.0.0-15134.gada1013c.el8.x86_64 112/138 2022-09-20T20:05:57.170 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:17.0.0-15134.gada1013c.el8.x86_64 112/138 2022-09-20T20:05:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:17.0.0-15134.gada1 113/138 2022-09-20T20:05:57.361 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: cephfs-mirror-2:17.0.0-15134.gada1013c.el8.x86_6 114/138 2022-09-20T20:05:57.379 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.0.0-15134.gada1 113/138 2022-09-20T20:05:57.510 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : rbd-mirror-2:17.0.0-15134.gada1013c.el8.x86_64 115/138 2022-09-20T20:05:57.546 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-mirror-2:17.0.0-15134.gada1013c.el8.x86_6 114/138 2022-09-20T20:05:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephfs-mirror-2:17.0.0-15134.gada1013c.el8.x86_6 114/138 2022-09-20T20:05:58.046 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:17.0.0-15134.gada1013c.el8.x86_64 115/138 2022-09-20T20:06:02.902 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: rbd-mirror-2:17.0.0-15134.gada1013c.el8.x86_64 115/138 2022-09-20T20:06:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:17.0.0-15134.gada1013c.el8.x86_64 115/138 2022-09-20T20:06:03.181 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-test-2:17.0.0-15134.gada1013c.el8.x86_64 116/138 2022-09-20T20:06:03.317 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/138 2022-09-20T20:06:03.417 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:17.0.0-15134.gada1013c.el8.x86_64 116/138 2022-09-20T20:06:03.521 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/138 2022-09-20T20:06:03.778 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 118/138 2022-09-20T20:06:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 117/138 2022-09-20T20:06:04.487 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/138 2022-09-20T20:06:04.657 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : ceph-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 119/138 2022-09-20T20:06:04.729 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/138 2022-09-20T20:06:04.861 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/138 2022-09-20T20:06:04.952 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 120/138 2022-09-20T20:06:05.048 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/138 2022-09-20T20:06:05.121 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 121/138 2022-09-20T20:06:05.279 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libcephfs-devel-2:17.0.0-15134.gada1013c.el8.x86 122/138 2022-09-20T20:06:05.307 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/138 2022-09-20T20:06:05.417 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-09-20T20:06:05.456 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:17.0.0-15134.gada1013c.el8.x86 122/138 2022-09-20T20:06:05.620 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-09-20T20:06:07.618 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-09-20T20:06:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-09-20T20:06:07.870 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-09-20T20:06:07.889 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-09-20T20:06:07.974 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-09-20T20:06:07.990 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-09-20T20:06:08.134 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : cephfs-top-2:17.0.0-15134.gada1013c.el8.noarch 125/138 2022-09-20T20:06:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-top-2:17.0.0-15134.gada1013c.el8.noarch 125/138 2022-09-20T20:06:08.333 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : rbd-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 126/138 2022-09-20T20:06:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 126/138 2022-09-20T20:06:08.597 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : rbd-nbd-2:17.0.0-15134.gada1013c.el8.x86_64 127/138 2022-09-20T20:06:08.710 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:17.0.0-15134.gada1013c.el8.x86_64 127/138 2022-09-20T20:06:08.973 INFO:teuthology.orchestra.run.smithi036.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-09-20T20:06:09.078 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-09-20T20:06:09.227 INFO:teuthology.orchestra.run.smithi036.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-09-20T20:06:09.374 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-7.el8 130/138 2022-09-20T20:06:09.383 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-7.el8 130/138 2022-09-20T20:06:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-09-20T20:06:09.553 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/138 2022-09-20T20:06:09.643 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/138 2022-09-20T20:06:09.717 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/138 2022-09-20T20:06:09.799 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/138 2022-09-20T20:06:09.879 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/138 2022-09-20T20:06:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/138 2022-09-20T20:06:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/138 2022-09-20T20:06:10.076 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-7.el8 132/138 2022-09-20T20:06:10.082 INFO:teuthology.orchestra.run.smithi036.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/138 2022-09-20T20:06:10.084 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-7.el8 132/138 2022-09-20T20:06:10.098 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/138 2022-09-20T20:06:10.254 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 133/138 2022-09-20T20:06:10.285 INFO:teuthology.orchestra.run.smithi036.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-09-20T20:06:10.285 INFO:teuthology.orchestra.run.smithi036.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-09-20T20:06:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-09-20T20:06:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-09-20T20:06:10.478 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-09-20T20:06:10.478 INFO:teuthology.orchestra.run.smithi036.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-09-20T20:06:10.663 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-09-20T20:06:10.664 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-09-20T20:06:10.728 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-09-20T20:06:10.728 INFO:teuthology.orchestra.run.smithi036.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-09-20T20:06:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-09-20T20:06:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-09-20T20:06:10.963 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-09-20T20:06:11.132 INFO:teuthology.orchestra.run.smithi036.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-09-20T20:06:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-09-20T20:06:11.367 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-09-20T20:06:11.712 INFO:teuthology.orchestra.run.smithi036.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-09-20T20:06:11.712 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-2:17.0.0-15134.gada1013c.el8.x86_64 1/138 2022-09-20T20:06:11.712 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-base-2:17.0.0-15134.gada1013c.el8.x86_64 2/138 2022-09-20T20:06:11.712 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 3/138 2022-09-20T20:06:11.713 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 4/138 2022-09-20T20:06:11.713 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-immutable-object-cache-2:17.0.0-15134.gada1 5/138 2022-09-20T20:06:11.713 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-mds-2:17.0.0-15134.gada1013c.el8.x86_64 6/138 2022-09-20T20:06:11.713 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-mgr-2:17.0.0-15134.gada1013c.el8.x86_64 7/138 2022-09-20T20:06:11.714 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-mon-2:17.0.0-15134.gada1013c.el8.x86_64 8/138 2022-09-20T20:06:11.714 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-osd-2:17.0.0-15134.gada1013c.el8.x86_64 9/138 2022-09-20T20:06:11.714 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-radosgw-2:17.0.0-15134.gada1013c.el8.x86_64 10/138 2022-09-20T20:06:11.714 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-selinux-2:17.0.0-15134.gada1013c.el8.x86_64 11/138 2022-09-20T20:06:11.715 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-test-2:17.0.0-15134.gada1013c.el8.x86_64 12/138 2022-09-20T20:06:11.715 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : cephfs-mirror-2:17.0.0-15134.gada1013c.el8.x86_6 13/138 2022-09-20T20:06:11.715 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libcephfs-devel-2:17.0.0-15134.gada1013c.el8.x86 14/138 2022-09-20T20:06:11.715 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libcephfs2-2:17.0.0-15134.gada1013c.el8.x86_64 15/138 2022-09-20T20:06:11.715 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libcephsqlite-2:17.0.0-15134.gada1013c.el8.x86_6 16/138 2022-09-20T20:06:11.716 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : librados-devel-2:17.0.0-15134.gada1013c.el8.x86_ 17/138 2022-09-20T20:06:11.716 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libradosstriper1-2:17.0.0-15134.gada1013c.el8.x8 18/138 2022-09-20T20:06:11.716 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : librgw2-2:17.0.0-15134.gada1013c.el8.x86_64 19/138 2022-09-20T20:06:11.716 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-ceph-argparse-2:17.0.0-15134.gada1013c.e 20/138 2022-09-20T20:06:11.717 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-ceph-common-2:17.0.0-15134.gada1013c.el8 21/138 2022-09-20T20:06:11.717 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-cephfs-2:17.0.0-15134.gada1013c.el8.x86_ 22/138 2022-09-20T20:06:11.717 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-rados-2:17.0.0-15134.gada1013c.el8.x86_6 23/138 2022-09-20T20:06:11.717 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-rbd-2:17.0.0-15134.gada1013c.el8.x86_64 24/138 2022-09-20T20:06:11.717 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-rgw-2:17.0.0-15134.gada1013c.el8.x86_64 25/138 2022-09-20T20:06:11.718 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : rbd-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 26/138 2022-09-20T20:06:11.718 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : rbd-mirror-2:17.0.0-15134.gada1013c.el8.x86_64 27/138 2022-09-20T20:06:11.718 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : rbd-nbd-2:17.0.0-15134.gada1013c.el8.x86_64 28/138 2022-09-20T20:06:11.718 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-grafana-dashboards-2:17.0.0-15134.gada1013c 29/138 2022-09-20T20:06:11.729 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-mgr-cephadm-2:17.0.0-15134.gada1013c.el8.no 30/138 2022-09-20T20:06:11.730 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-mgr-dashboard-2:17.0.0-15134.gada1013c.el8. 31/138 2022-09-20T20:06:11.730 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.0.0-15134.gad 32/138 2022-09-20T20:06:11.730 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-mgr-modules-core-2:17.0.0-15134.gada1013c.e 33/138 2022-09-20T20:06:11.730 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-mgr-rook-2:17.0.0-15134.gada1013c.el8.noarc 34/138 2022-09-20T20:06:11.730 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-prometheus-alerts-2:17.0.0-15134.gada1013c. 35/138 2022-09-20T20:06:11.731 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : ceph-volume-2:17.0.0-15134.gada1013c.el8.noarch 36/138 2022-09-20T20:06:11.731 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : cephadm-2:17.0.0-15134.gada1013c.el8.noarch 37/138 2022-09-20T20:06:11.731 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : cephfs-top-2:17.0.0-15134.gada1013c.el8.noarch 38/138 2022-09-20T20:06:11.731 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-09-20T20:06:11.732 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-09-20T20:06:11.732 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-09-20T20:06:11.732 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-09-20T20:06:11.733 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-09-20T20:06:11.733 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-09-20T20:06:11.733 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-09-20T20:06:11.733 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-09-20T20:06:11.734 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-09-20T20:06:11.734 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-09-20T20:06:11.734 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-09-20T20:06:11.734 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-09-20T20:06:11.735 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-09-20T20:06:11.735 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-09-20T20:06:11.735 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-09-20T20:06:11.735 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-09-20T20:06:11.736 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-09-20T20:06:11.736 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-09-20T20:06:11.736 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-09-20T20:06:11.736 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-09-20T20:06:11.736 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-09-20T20:06:11.737 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-09-20T20:06:11.737 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-09-20T20:06:11.737 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-09-20T20:06:11.737 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-09-20T20:06:11.738 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-09-20T20:06:11.743 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-09-20T20:06:11.744 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-09-20T20:06:11.744 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-09-20T20:06:11.744 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-09-20T20:06:11.745 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-09-20T20:06:11.745 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-09-20T20:06:11.745 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-09-20T20:06:11.745 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 72/138 2022-09-20T20:06:11.745 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : bison-3.0.4-10.el8.x86_64 73/138 2022-09-20T20:06:11.746 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 74/138 2022-09-20T20:06:11.746 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 75/138 2022-09-20T20:06:11.746 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 76/138 2022-09-20T20:06:11.746 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/138 2022-09-20T20:06:11.747 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 78/138 2022-09-20T20:06:11.747 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 79/138 2022-09-20T20:06:11.747 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 80/138 2022-09-20T20:06:11.747 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 81/138 2022-09-20T20:06:11.747 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 82/138 2022-09-20T20:06:11.748 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 83/138 2022-09-20T20:06:11.748 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 84/138 2022-09-20T20:06:11.748 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 85/138 2022-09-20T20:06:11.748 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/138 2022-09-20T20:06:11.749 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 87/138 2022-09-20T20:06:11.749 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/138 2022-09-20T20:06:11.749 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : flex-2.6.1-9.el8.x86_64 89/138 2022-09-20T20:06:11.749 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 90/138 2022-09-20T20:06:11.749 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : jq-1.6-3.el8.x86_64 91/138 2022-09-20T20:06:11.750 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 92/138 2022-09-20T20:06:11.750 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : automake-1.16.1-7.el8.noarch 93/138 2022-09-20T20:06:11.750 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 94/138 2022-09-20T20:06:11.750 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 95/138 2022-09-20T20:06:11.751 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 96/138 2022-09-20T20:06:11.751 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libpq-13.5-1.el8.x86_64 97/138 2022-09-20T20:06:11.751 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 98/138 2022-09-20T20:06:11.751 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 99/138 2022-09-20T20:06:11.752 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 100/138 2022-09-20T20:06:11.752 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 101/138 2022-09-20T20:06:11.752 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/138 2022-09-20T20:06:11.752 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 103/138 2022-09-20T20:06:11.752 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 104/138 2022-09-20T20:06:11.753 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 105/138 2022-09-20T20:06:11.753 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/138 2022-09-20T20:06:11.753 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 107/138 2022-09-20T20:06:11.753 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 108/138 2022-09-20T20:06:11.754 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 109/138 2022-09-20T20:06:11.754 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 110/138 2022-09-20T20:06:11.754 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 111/138 2022-09-20T20:06:11.754 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 112/138 2022-09-20T20:06:11.755 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 113/138 2022-09-20T20:06:11.755 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 114/138 2022-09-20T20:06:11.755 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 115/138 2022-09-20T20:06:11.755 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/138 2022-09-20T20:06:11.756 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 117/138 2022-09-20T20:06:11.756 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/138 2022-09-20T20:06:11.756 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 119/138 2022-09-20T20:06:11.756 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 120/138 2022-09-20T20:06:11.757 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-7.el8 121/138 2022-09-20T20:06:11.757 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 122/138 2022-09-20T20:06:11.757 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 123/138 2022-09-20T20:06:11.757 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 124/138 2022-09-20T20:06:11.757 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/138 2022-09-20T20:06:11.758 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 126/138 2022-09-20T20:06:11.758 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 127/138 2022-09-20T20:06:11.758 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 128/138 2022-09-20T20:06:11.758 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : librados2-2:17.0.0-15134.gada1013c.el8.x86_64 129/138 2022-09-20T20:06:11.759 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-09-20T20:06:11.759 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : librbd1-2:17.0.0-15134.gada1013c.el8.x86_64 131/138 2022-09-20T20:06:11.759 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-09-20T20:06:11.759 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-09-20T20:06:11.760 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-09-20T20:06:11.760 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-09-20T20:06:11.760 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-09-20T20:06:11.761 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-09-20T20:06:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-09-20T20:06:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:17.0.0-15134.gada1013c.el8.x86_64 1/138 2022-09-20T20:06:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:17.0.0-15134.gada1013c.el8.x86_64 2/138 2022-09-20T20:06:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 3/138 2022-09-20T20:06:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 4/138 2022-09-20T20:06:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:17.0.0-15134.gada1 5/138 2022-09-20T20:06:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:17.0.0-15134.gada1013c.el8.x86_64 6/138 2022-09-20T20:06:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:17.0.0-15134.gada1013c.el8.x86_64 7/138 2022-09-20T20:06:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:17.0.0-15134.gada1013c.el8.x86_64 8/138 2022-09-20T20:06:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:17.0.0-15134.gada1013c.el8.x86_64 9/138 2022-09-20T20:06:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:17.0.0-15134.gada1013c.el8.x86_64 10/138 2022-09-20T20:06:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:17.0.0-15134.gada1013c.el8.x86_64 11/138 2022-09-20T20:06:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:17.0.0-15134.gada1013c.el8.x86_64 12/138 2022-09-20T20:06:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-mirror-2:17.0.0-15134.gada1013c.el8.x86_6 13/138 2022-09-20T20:06:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:17.0.0-15134.gada1013c.el8.x86 14/138 2022-09-20T20:06:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:17.0.0-15134.gada1013c.el8.x86_64 15/138 2022-09-20T20:06:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:17.0.0-15134.gada1013c.el8.x86_6 16/138 2022-09-20T20:06:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:17.0.0-15134.gada1013c.el8.x86_ 17/138 2022-09-20T20:06:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:17.0.0-15134.gada1013c.el8.x8 18/138 2022-09-20T20:06:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:17.0.0-15134.gada1013c.el8.x86_64 19/138 2022-09-20T20:06:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:17.0.0-15134.gada1013c.e 20/138 2022-09-20T20:06:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:17.0.0-15134.gada1013c.el8 21/138 2022-09-20T20:06:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:17.0.0-15134.gada1013c.el8.x86_ 22/138 2022-09-20T20:06:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:17.0.0-15134.gada1013c.el8.x86_6 23/138 2022-09-20T20:06:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:17.0.0-15134.gada1013c.el8.x86_64 24/138 2022-09-20T20:06:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:17.0.0-15134.gada1013c.el8.x86_64 25/138 2022-09-20T20:06:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 26/138 2022-09-20T20:06:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:17.0.0-15134.gada1013c.el8.x86_64 27/138 2022-09-20T20:06:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:17.0.0-15134.gada1013c.el8.x86_64 28/138 2022-09-20T20:06:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:17.0.0-15134.gada1013c 29/138 2022-09-20T20:06:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:17.0.0-15134.gada1013c.el8.no 30/138 2022-09-20T20:06:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:17.0.0-15134.gada1013c.el8. 31/138 2022-09-20T20:06:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.0.0-15134.gad 32/138 2022-09-20T20:06:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:17.0.0-15134.gada1013c.e 33/138 2022-09-20T20:06:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:17.0.0-15134.gada1013c.el8.noarc 34/138 2022-09-20T20:06:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:17.0.0-15134.gada1013c. 35/138 2022-09-20T20:06:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:17.0.0-15134.gada1013c.el8.noarch 36/138 2022-09-20T20:06:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:17.0.0-15134.gada1013c.el8.noarch 37/138 2022-09-20T20:06:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-top-2:17.0.0-15134.gada1013c.el8.noarch 38/138 2022-09-20T20:06:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-09-20T20:06:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-09-20T20:06:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-09-20T20:06:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-09-20T20:06:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-09-20T20:06:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-09-20T20:06:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-09-20T20:06:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-09-20T20:06:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-09-20T20:06:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-09-20T20:06:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-09-20T20:06:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-09-20T20:06:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-09-20T20:06:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-09-20T20:06:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-09-20T20:06:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-09-20T20:06:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-09-20T20:06:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-09-20T20:06:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-09-20T20:06:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-09-20T20:06:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-09-20T20:06:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-09-20T20:06:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-09-20T20:06:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-09-20T20:06:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-09-20T20:06:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-09-20T20:06:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-09-20T20:06:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-09-20T20:06:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-09-20T20:06:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-09-20T20:06:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-09-20T20:06:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-09-20T20:06:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-09-20T20:06:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 72/138 2022-09-20T20:06:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 73/138 2022-09-20T20:06:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 74/138 2022-09-20T20:06:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/138 2022-09-20T20:06:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 76/138 2022-09-20T20:06:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 77/138 2022-09-20T20:06:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 78/138 2022-09-20T20:06:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 79/138 2022-09-20T20:06:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 80/138 2022-09-20T20:06:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 81/138 2022-09-20T20:06:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/138 2022-09-20T20:06:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 83/138 2022-09-20T20:06:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 84/138 2022-09-20T20:06:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/138 2022-09-20T20:06:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 86/138 2022-09-20T20:06:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/138 2022-09-20T20:06:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 88/138 2022-09-20T20:06:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 89/138 2022-09-20T20:06:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-7.el8 90/138 2022-09-20T20:06:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 91/138 2022-09-20T20:06:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 92/138 2022-09-20T20:06:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 93/138 2022-09-20T20:06:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/138 2022-09-20T20:06:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 95/138 2022-09-20T20:06:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 96/138 2022-09-20T20:06:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 97/138 2022-09-20T20:06:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 98/138 2022-09-20T20:06:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : bison-3.0.4-10.el8.x86_64 99/138 2022-09-20T20:06:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 100/138 2022-09-20T20:06:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 101/138 2022-09-20T20:06:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 102/138 2022-09-20T20:06:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 103/138 2022-09-20T20:06:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/138 2022-09-20T20:06:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 105/138 2022-09-20T20:06:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 106/138 2022-09-20T20:06:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 107/138 2022-09-20T20:06:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/138 2022-09-20T20:06:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 109/138 2022-09-20T20:06:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 110/138 2022-09-20T20:06:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 111/138 2022-09-20T20:06:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/138 2022-09-20T20:06:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/138 2022-09-20T20:06:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 114/138 2022-09-20T20:06:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flex-2.6.1-9.el8.x86_64 115/138 2022-09-20T20:06:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 116/138 2022-09-20T20:06:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : jq-1.6-3.el8.x86_64 117/138 2022-09-20T20:06:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 118/138 2022-09-20T20:06:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : automake-1.16.1-7.el8.noarch 119/138 2022-09-20T20:06:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 120/138 2022-09-20T20:06:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 121/138 2022-09-20T20:06:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 122/138 2022-09-20T20:06:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/138 2022-09-20T20:06:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 124/138 2022-09-20T20:06:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 125/138 2022-09-20T20:06:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 126/138 2022-09-20T20:06:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 127/138 2022-09-20T20:06:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 128/138 2022-09-20T20:06:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:17.0.0-15134.gada1013c.el8.x86_64 129/138 2022-09-20T20:06:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-09-20T20:06:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:17.0.0-15134.gada1013c.el8.x86_64 131/138 2022-09-20T20:06:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-09-20T20:06:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-09-20T20:06:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-09-20T20:06:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-09-20T20:06:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-09-20T20:06:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-09-20T20:06:13.587 INFO:teuthology.orchestra.run.smithi036.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-09-20T20:06:13.588 INFO:teuthology.orchestra.run.smithi036.stdout:Installed products updated. 2022-09-20T20:06:13.834 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:13.835 INFO:teuthology.orchestra.run.smithi036.stdout:Upgraded: 2022-09-20T20:06:13.835 INFO:teuthology.orchestra.run.smithi036.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-09-20T20:06:13.835 INFO:teuthology.orchestra.run.smithi036.stdout: librados2-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.835 INFO:teuthology.orchestra.run.smithi036.stdout: librbd1-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.836 INFO:teuthology.orchestra.run.smithi036.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-09-20T20:06:13.836 INFO:teuthology.orchestra.run.smithi036.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-09-20T20:06:13.836 INFO:teuthology.orchestra.run.smithi036.stdout:Installed: 2022-09-20T20:06:13.836 INFO:teuthology.orchestra.run.smithi036.stdout: NetworkManager-initscripts-updown-1:1.36.0-7.el8_6.noarch 2022-09-20T20:06:13.837 INFO:teuthology.orchestra.run.smithi036.stdout: automake-1.16.1-7.el8.noarch 2022-09-20T20:06:13.837 INFO:teuthology.orchestra.run.smithi036.stdout: bison-3.0.4-10.el8.x86_64 2022-09-20T20:06:13.837 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.837 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-base-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.837 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.838 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.838 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-grafana-dashboards-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:13.838 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-immutable-object-cache-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.838 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mds-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.839 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.839 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-cephadm-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:13.839 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-dashboard-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:13.839 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-diskprediction-local-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:13.839 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-modules-core-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:13.840 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mgr-rook-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:13.840 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-mon-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.840 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-osd-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.840 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-prometheus-alerts-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:13.840 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-radosgw-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.841 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-selinux-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.841 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-test-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.841 INFO:teuthology.orchestra.run.smithi036.stdout: ceph-volume-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:13.841 INFO:teuthology.orchestra.run.smithi036.stdout: cephadm-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:13.841 INFO:teuthology.orchestra.run.smithi036.stdout: cephfs-mirror-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.842 INFO:teuthology.orchestra.run.smithi036.stdout: cephfs-top-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:13.842 INFO:teuthology.orchestra.run.smithi036.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-09-20T20:06:13.842 INFO:teuthology.orchestra.run.smithi036.stdout: flex-2.6.1-9.el8.x86_64 2022-09-20T20:06:13.842 INFO:teuthology.orchestra.run.smithi036.stdout: fuse-2.9.7-15.el8.x86_64 2022-09-20T20:06:13.842 INFO:teuthology.orchestra.run.smithi036.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-09-20T20:06:13.843 INFO:teuthology.orchestra.run.smithi036.stdout: jq-1.6-3.el8.x86_64 2022-09-20T20:06:13.843 INFO:teuthology.orchestra.run.smithi036.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-09-20T20:06:13.843 INFO:teuthology.orchestra.run.smithi036.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-09-20T20:06:13.844 INFO:teuthology.orchestra.run.smithi036.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-09-20T20:06:13.844 INFO:teuthology.orchestra.run.smithi036.stdout: libcephfs-devel-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.844 INFO:teuthology.orchestra.run.smithi036.stdout: libcephfs2-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.844 INFO:teuthology.orchestra.run.smithi036.stdout: libcephsqlite-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.844 INFO:teuthology.orchestra.run.smithi036.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-09-20T20:06:13.845 INFO:teuthology.orchestra.run.smithi036.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-09-20T20:06:13.845 INFO:teuthology.orchestra.run.smithi036.stdout: liboath-2.6.2-3.el8.x86_64 2022-09-20T20:06:13.845 INFO:teuthology.orchestra.run.smithi036.stdout: libpq-13.5-1.el8.x86_64 2022-09-20T20:06:13.845 INFO:teuthology.orchestra.run.smithi036.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-09-20T20:06:13.845 INFO:teuthology.orchestra.run.smithi036.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-09-20T20:06:13.846 INFO:teuthology.orchestra.run.smithi036.stdout: librados-devel-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.846 INFO:teuthology.orchestra.run.smithi036.stdout: libradosstriper1-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.846 INFO:teuthology.orchestra.run.smithi036.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-09-20T20:06:13.846 INFO:teuthology.orchestra.run.smithi036.stdout: librgw2-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.847 INFO:teuthology.orchestra.run.smithi036.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-09-20T20:06:13.847 INFO:teuthology.orchestra.run.smithi036.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-09-20T20:06:13.847 INFO:teuthology.orchestra.run.smithi036.stdout: libtool-2.4.6-25.el8.x86_64 2022-09-20T20:06:13.847 INFO:teuthology.orchestra.run.smithi036.stdout: libunwind-1.3.1-3.el8.x86_64 2022-09-20T20:06:13.847 INFO:teuthology.orchestra.run.smithi036.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-09-20T20:06:13.848 INFO:teuthology.orchestra.run.smithi036.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-09-20T20:06:13.848 INFO:teuthology.orchestra.run.smithi036.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-09-20T20:06:13.848 INFO:teuthology.orchestra.run.smithi036.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-09-20T20:06:13.848 INFO:teuthology.orchestra.run.smithi036.stdout: openblas-0.3.15-3.el8.x86_64 2022-09-20T20:06:13.848 INFO:teuthology.orchestra.run.smithi036.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-09-20T20:06:13.849 INFO:teuthology.orchestra.run.smithi036.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-09-20T20:06:13.849 INFO:teuthology.orchestra.run.smithi036.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-09-20T20:06:13.849 INFO:teuthology.orchestra.run.smithi036.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-09-20T20:06:13.849 INFO:teuthology.orchestra.run.smithi036.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-09-20T20:06:13.850 INFO:teuthology.orchestra.run.smithi036.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-09-20T20:06:13.850 INFO:teuthology.orchestra.run.smithi036.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-09-20T20:06:13.850 INFO:teuthology.orchestra.run.smithi036.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-09-20T20:06:13.850 INFO:teuthology.orchestra.run.smithi036.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-09-20T20:06:13.850 INFO:teuthology.orchestra.run.smithi036.stdout: python3-babel-2.5.1-7.el8.noarch 2022-09-20T20:06:13.851 INFO:teuthology.orchestra.run.smithi036.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-09-20T20:06:13.851 INFO:teuthology.orchestra.run.smithi036.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-09-20T20:06:13.851 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-09-20T20:06:13.851 INFO:teuthology.orchestra.run.smithi036.stdout: python3-ceph-argparse-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.852 INFO:teuthology.orchestra.run.smithi036.stdout: python3-ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.852 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cephfs-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.852 INFO:teuthology.orchestra.run.smithi036.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-09-20T20:06:13.852 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-09-20T20:06:13.852 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-09-20T20:06:13.853 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-09-20T20:06:13.853 INFO:teuthology.orchestra.run.smithi036.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-09-20T20:06:13.853 INFO:teuthology.orchestra.run.smithi036.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-09-20T20:06:13.853 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jaraco-6.2-6.el8.noarch 2022-09-20T20:06:13.853 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-09-20T20:06:13.854 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-09-20T20:06:13.854 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-09-20T20:06:13.854 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-09-20T20:06:13.854 INFO:teuthology.orchestra.run.smithi036.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-09-20T20:06:13.854 INFO:teuthology.orchestra.run.smithi036.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-09-20T20:06:13.855 INFO:teuthology.orchestra.run.smithi036.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-09-20T20:06:13.855 INFO:teuthology.orchestra.run.smithi036.stdout: python3-mako-1.0.6-13.el8.noarch 2022-09-20T20:06:13.855 INFO:teuthology.orchestra.run.smithi036.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-09-20T20:06:13.855 INFO:teuthology.orchestra.run.smithi036.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-09-20T20:06:13.856 INFO:teuthology.orchestra.run.smithi036.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-09-20T20:06:13.856 INFO:teuthology.orchestra.run.smithi036.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-09-20T20:06:13.856 INFO:teuthology.orchestra.run.smithi036.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-09-20T20:06:13.856 INFO:teuthology.orchestra.run.smithi036.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-09-20T20:06:13.856 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-09-20T20:06:13.857 INFO:teuthology.orchestra.run.smithi036.stdout: python3-portend-2.6-1.el8.noarch 2022-09-20T20:06:13.857 INFO:teuthology.orchestra.run.smithi036.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-09-20T20:06:13.857 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-09-20T20:06:13.857 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-09-20T20:06:13.858 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-09-20T20:06:13.858 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pycparser-2.14-14.el8.noarch 2022-09-20T20:06:13.858 INFO:teuthology.orchestra.run.smithi036.stdout: python3-pytz-2017.2-9.el8.noarch 2022-09-20T20:06:13.858 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rados-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.858 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rbd-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.859 INFO:teuthology.orchestra.run.smithi036.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-09-20T20:06:13.859 INFO:teuthology.orchestra.run.smithi036.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-09-20T20:06:13.859 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rgw-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.859 INFO:teuthology.orchestra.run.smithi036.stdout: python3-routes-2.4.1-12.el8.noarch 2022-09-20T20:06:13.859 INFO:teuthology.orchestra.run.smithi036.stdout: python3-rsa-4.8-1.el8.noarch 2022-09-20T20:06:13.860 INFO:teuthology.orchestra.run.smithi036.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-09-20T20:06:13.860 INFO:teuthology.orchestra.run.smithi036.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-09-20T20:06:13.860 INFO:teuthology.orchestra.run.smithi036.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-09-20T20:06:13.860 INFO:teuthology.orchestra.run.smithi036.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-09-20T20:06:13.860 INFO:teuthology.orchestra.run.smithi036.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-09-20T20:06:13.861 INFO:teuthology.orchestra.run.smithi036.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-09-20T20:06:13.861 INFO:teuthology.orchestra.run.smithi036.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-09-20T20:06:13.861 INFO:teuthology.orchestra.run.smithi036.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-09-20T20:06:13.861 INFO:teuthology.orchestra.run.smithi036.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-09-20T20:06:13.862 INFO:teuthology.orchestra.run.smithi036.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-09-20T20:06:13.862 INFO:teuthology.orchestra.run.smithi036.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-09-20T20:06:13.862 INFO:teuthology.orchestra.run.smithi036.stdout: rbd-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.862 INFO:teuthology.orchestra.run.smithi036.stdout: rbd-mirror-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.863 INFO:teuthology.orchestra.run.smithi036.stdout: rbd-nbd-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:13.863 INFO:teuthology.orchestra.run.smithi036.stdout: socat-1.7.4.1-1.el8.x86_64 2022-09-20T20:06:13.863 INFO:teuthology.orchestra.run.smithi036.stdout: thrift-0.13.0-2.el8.x86_64 2022-09-20T20:06:13.863 INFO:teuthology.orchestra.run.smithi036.stdout: uuid-1.6.2-43.el8.x86_64 2022-09-20T20:06:13.863 INFO:teuthology.orchestra.run.smithi036.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-09-20T20:06:13.864 INFO:teuthology.orchestra.run.smithi036.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-09-20T20:06:13.864 INFO:teuthology.orchestra.run.smithi036.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-09-20T20:06:13.864 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:13.865 INFO:teuthology.orchestra.run.smithi036.stdout:Complete! 2022-09-20T20:06:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-09-20T20:06:13.955 INFO:teuthology.orchestra.run.smithi115.stdout:Installed products updated. 2022-09-20T20:06:14.010 DEBUG:teuthology.parallel:result is None 2022-09-20T20:06:14.203 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:14.204 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2022-09-20T20:06:14.204 INFO:teuthology.orchestra.run.smithi115.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-09-20T20:06:14.204 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.204 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.205 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-09-20T20:06:14.205 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-09-20T20:06:14.205 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2022-09-20T20:06:14.205 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-initscripts-updown-1:1.36.0-7.el8_6.noarch 2022-09-20T20:06:14.206 INFO:teuthology.orchestra.run.smithi115.stdout: automake-1.16.1-7.el8.noarch 2022-09-20T20:06:14.206 INFO:teuthology.orchestra.run.smithi115.stdout: bison-3.0.4-10.el8.x86_64 2022-09-20T20:06:14.206 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.206 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.207 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.207 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.207 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:14.207 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.207 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.208 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.208 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:14.208 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:14.208 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:14.209 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:14.209 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:14.209 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.209 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.209 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:14.210 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.210 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.210 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.210 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:14.210 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:14.211 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.211 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top-2:17.0.0-15134.gada1013c.el8.noarch 2022-09-20T20:06:14.211 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-09-20T20:06:14.211 INFO:teuthology.orchestra.run.smithi115.stdout: flex-2.6.1-9.el8.x86_64 2022-09-20T20:06:14.212 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-2.9.7-15.el8.x86_64 2022-09-20T20:06:14.212 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-09-20T20:06:14.212 INFO:teuthology.orchestra.run.smithi115.stdout: jq-1.6-3.el8.x86_64 2022-09-20T20:06:14.212 INFO:teuthology.orchestra.run.smithi115.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-09-20T20:06:14.212 INFO:teuthology.orchestra.run.smithi115.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-09-20T20:06:14.212 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-09-20T20:06:14.213 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.213 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.213 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.213 INFO:teuthology.orchestra.run.smithi115.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-09-20T20:06:14.213 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-09-20T20:06:14.214 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.2-3.el8.x86_64 2022-09-20T20:06:14.214 INFO:teuthology.orchestra.run.smithi115.stdout: libpq-13.5-1.el8.x86_64 2022-09-20T20:06:14.214 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-09-20T20:06:14.214 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-09-20T20:06:14.215 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.215 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.215 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-09-20T20:06:14.215 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.215 INFO:teuthology.orchestra.run.smithi115.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-09-20T20:06:14.216 INFO:teuthology.orchestra.run.smithi115.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-09-20T20:06:14.216 INFO:teuthology.orchestra.run.smithi115.stdout: libtool-2.4.6-25.el8.x86_64 2022-09-20T20:06:14.216 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.3.1-3.el8.x86_64 2022-09-20T20:06:14.216 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-09-20T20:06:14.216 INFO:teuthology.orchestra.run.smithi115.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-09-20T20:06:14.217 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-09-20T20:06:14.217 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-09-20T20:06:14.217 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.15-3.el8.x86_64 2022-09-20T20:06:14.217 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-09-20T20:06:14.218 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-09-20T20:06:14.218 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-09-20T20:06:14.218 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-09-20T20:06:14.218 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-09-20T20:06:14.218 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-09-20T20:06:14.219 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-09-20T20:06:14.219 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-09-20T20:06:14.219 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-09-20T20:06:14.219 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.5.1-7.el8.noarch 2022-09-20T20:06:14.219 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-09-20T20:06:14.220 INFO:teuthology.orchestra.run.smithi115.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-09-20T20:06:14.220 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-09-20T20:06:14.220 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.220 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.220 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.221 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-09-20T20:06:14.221 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-09-20T20:06:14.221 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-09-20T20:06:14.221 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-09-20T20:06:14.221 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-09-20T20:06:14.222 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-09-20T20:06:14.222 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-6.2-6.el8.noarch 2022-09-20T20:06:14.222 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-09-20T20:06:14.222 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-09-20T20:06:14.222 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-09-20T20:06:14.223 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-09-20T20:06:14.223 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-09-20T20:06:14.223 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-09-20T20:06:14.223 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-09-20T20:06:14.223 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.0.6-13.el8.noarch 2022-09-20T20:06:14.224 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-09-20T20:06:14.224 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-09-20T20:06:14.224 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-09-20T20:06:14.224 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-09-20T20:06:14.224 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-09-20T20:06:14.225 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-09-20T20:06:14.225 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-09-20T20:06:14.225 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-2.6-1.el8.noarch 2022-09-20T20:06:14.225 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-09-20T20:06:14.226 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-09-20T20:06:14.226 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-09-20T20:06:14.226 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-09-20T20:06:14.226 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.14-14.el8.noarch 2022-09-20T20:06:14.226 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2017.2-9.el8.noarch 2022-09-20T20:06:14.227 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.227 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.227 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-09-20T20:06:14.227 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-09-20T20:06:14.227 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.228 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.4.1-12.el8.noarch 2022-09-20T20:06:14.228 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.8-1.el8.noarch 2022-09-20T20:06:14.228 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-09-20T20:06:14.228 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-09-20T20:06:14.228 INFO:teuthology.orchestra.run.smithi115.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-09-20T20:06:14.229 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-09-20T20:06:14.229 INFO:teuthology.orchestra.run.smithi115.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-09-20T20:06:14.229 INFO:teuthology.orchestra.run.smithi115.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-09-20T20:06:14.229 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-09-20T20:06:14.229 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-09-20T20:06:14.230 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-09-20T20:06:14.230 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-09-20T20:06:14.230 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-09-20T20:06:14.230 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.231 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.231 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:17.0.0-15134.gada1013c.el8.x86_64 2022-09-20T20:06:14.231 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-1.el8.x86_64 2022-09-20T20:06:14.231 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.13.0-2.el8.x86_64 2022-09-20T20:06:14.231 INFO:teuthology.orchestra.run.smithi115.stdout: uuid-1.6.2-43.el8.x86_64 2022-09-20T20:06:14.231 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-09-20T20:06:14.232 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-09-20T20:06:14.232 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-09-20T20:06:14.233 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:14.233 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2022-09-20T20:06:14.399 DEBUG:teuthology.parallel:result is None 2022-09-20T20:06:14.400 DEBUG:teuthology.orchestra.run.smithi036:> sudo lsb_release -is 2022-09-20T20:06:14.462 INFO:teuthology.orchestra.run.smithi036.stdout:RedHatEnterprise 2022-09-20T20:06:14.463 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-09-20T20:06:14.464 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ada1013c357729fc3bf91fdc0bbe1d411f5af166 2022-09-20T20:06:14.669 DEBUG:teuthology.orchestra.run.smithi036:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-09-20T20:06:14.692 INFO:teuthology.orchestra.run.smithi036.stdout:17.0.0-15134.gada1013c.el8 2022-09-20T20:06:14.693 INFO:teuthology.packaging:The installed version of ceph is 17.0.0-15134.gada1013c.el8 2022-09-20T20:06:14.693 INFO:teuthology.task.install:The correct ceph version 17.0.0-15134.gada1013c is installed. 2022-09-20T20:06:14.694 DEBUG:teuthology.orchestra.run.smithi115:> sudo lsb_release -is 2022-09-20T20:06:14.748 INFO:teuthology.orchestra.run.smithi115.stdout:RedHatEnterprise 2022-09-20T20:06:14.750 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-09-20T20:06:14.750 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ada1013c357729fc3bf91fdc0bbe1d411f5af166 2022-09-20T20:06:14.949 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-09-20T20:06:14.971 INFO:teuthology.orchestra.run.smithi115.stdout:17.0.0-15134.gada1013c.el8 2022-09-20T20:06:14.971 INFO:teuthology.packaging:The installed version of ceph is 17.0.0-15134.gada1013c.el8 2022-09-20T20:06:14.972 INFO:teuthology.task.install:The correct ceph version 17.0.0-15134.gada1013c is installed. 2022-09-20T20:06:14.972 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-09-20T20:06:14.973 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:14.973 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-09-20T20:06:15.002 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:15.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-09-20T20:06:15.039 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-09-20T20:06:15.039 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:15.040 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/usr/bin/daemon-helper 2022-09-20T20:06:15.070 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-09-20T20:06:15.138 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:15.138 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2022-09-20T20:06:15.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-09-20T20:06:15.232 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-09-20T20:06:15.232 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:15.233 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/usr/bin/adjust-ulimits 2022-09-20T20:06:15.261 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-09-20T20:06:15.329 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:15.330 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2022-09-20T20:06:15.356 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-09-20T20:06:15.422 INFO:teuthology.run_tasks:Running task ceph... 2022-09-20T20:06:15.483 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-09-20T20:06:15.484 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 777 /var/log/ceph 2022-09-20T20:06:15.487 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /var/log/ceph 2022-09-20T20:06:15.513 INFO:tasks.ceph:Disabling ceph logrotate... 2022-09-20T20:06:15.514 DEBUG:teuthology.orchestra.run.smithi036:> sudo rm -f -- /etc/logrotate.d/ceph 2022-09-20T20:06:15.555 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/logrotate.d/ceph 2022-09-20T20:06:15.581 INFO:tasks.ceph:Creating extra log directories... 2022-09-20T20:06:15.582 DEBUG:teuthology.orchestra.run.smithi036:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-09-20T20:06:15.623 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-09-20T20:06:15.656 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-09-20T20:06:15.657 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-09-20T20:06:15.657 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-09-20T20:06:15.658 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:15.658 DEBUG:teuthology.orchestra.run.smithi036:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-09-20T20:06:15.706 DEBUG:teuthology.orchestra.run.smithi036:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-09-20T20:06:15.821 DEBUG:teuthology.orchestra.run.smithi036:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-09-20T20:06:15.849 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:15.849 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-09-20T20:06:15.866 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-09-20T20:06:15.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-09-20T20:06:16.009 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-09-20T20:06:16.010 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-09-20T20:06:16.010 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2022-09-20_19:41:25-fs:functional-wip-pdonnell-testing-20220920.151129-distro-default-smithi/7039471', 'branch': 'wip-pdonnell-testing-20220920.151129', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7039471', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2022-09-20_19:41:25-fs:functional-wip-pdonnell-testing-20220920.151129-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20220920.151129'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'sha1': 'ada1013c357729fc3bf91fdc0bbe1d411f5af166'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ada1013c357729fc3bf91fdc0bbe1d411f5af166'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20220920.151129', 'sha1': '7874f3907d0b2e2027e668918a6854269060504f'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 50, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'ada1013c357729fc3bf91fdc0bbe1d411f5af166', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-pdonnell-testing-20220920.151129', 'suite_path': '/home/teuthworker/src/github.com_batrick_ceph_7874f3907d0b2e2027e668918a6854269060504f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/batrick/ceph.git', 'suite_sha1': '7874f3907d0b2e2027e668918a6854269060504f', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi036.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb7XvCj6ppza8aYF22opmyta1A2KNqYjUn8APLjXM6u4BOCCwjrDMJ2YRUq0pwsj6GqrFcpKh33X1Wycqe+EsHk/wZGitUexLIrdvcgu4K7+VkTRkssYrqIaEwT1GSRyLF667txFJpg/UAN6FBck97qimlM5eEAdoAtRClciIRQZIUlt8PDSp3ZzU+/Y6jbwxtREp0WG/DWUOroPY25j3vHwYnjBlHyCjR84aVxf9EqjQMCYPOPwbblf6iAfsSvVKPTpbS0Qv17mE9MCZI0IIQrTgrJdCDZtk+BBz0Rn0K2jX0C8CdCzWH8eVlv8kAL+ZLAMEcH9+jxwGU20tG71/r6HPCYCX2CbZmeUnkRwl5hPKd1RJV//ae/k6tF8weeR5f1FcAgXAkzqvnhLjutDHS+nhqN3ggVfHxdh7Mt9Zyo+iJBz1ijD9ncHrPlsdWrYnHqFJ+e4hl3NnWNU6vrFNXcp3ks/oPF6RoxvCBma10LKE6thbcld81LHlbZVrNFnk=', 'smithi115.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDS0vHkhkc7dtzk8QNT7mI8egs4xNS98blr0rI/LG0d9zDyaTqxeVFnQcKSplSy4wcAkBfu87Px4ePWEB7C/dB4bpKS+gyunsdbN4lfmVs1ce84iUMrt/LgZvIdFqmG6fXCYaCdqAZCK1CLGjvydAGz1jgwadrAprI3zPNNAIOuFh4bQsvVj+tbX9hJ5P8zfb1mtGzwZQc7iwu0mEwBmmbQlEGOFI258soNPFGT9QgVoZKgHUvms+4PCEOW1k6sVCPmfsZI5Jc+GOnf8ElIRlFImgY1yPHehvKeSu9G37XVWlqc/2SpGW/7evNSqK4NLzO9L1QaO/2fLMZL/PEyfSdLJakCIT4QcdnJHYufWx4VuAeyhWOAzVcCMdSes3dcmDYhAaZB3Tg9zreDT7Pi/y9SupY8+UrGAcVnsQY9QxhQHE23jbAUUU01E3078MoDqS5WAsi3Vtrc0Hom315zY+vO30CkFGrTQo0OeBrc4mHZ7V3FIUag7NlPF6UN/VLY2Gk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ada1013c357729fc3bf91fdc0bbe1d411f5af166'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_recovery_pool']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b58f2c18636eb10faa77ed3614abd00cb85dfc2c', 'timestamp': '2022-09-20_19:41:25', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.14413'} 2022-09-20T20:06:16.011 DEBUG:teuthology.orchestra.run.smithi036:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-09-20T20:06:16.014 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-09-20T20:06:16.061 DEBUG:teuthology.orchestra.run.smithi036:> sudo install -d -m0777 -- /var/run/ceph 2022-09-20T20:06:16.071 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/run/ceph 2022-09-20T20:06:16.129 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:16.130 DEBUG:teuthology.orchestra.run.smithi036:> dd if=/scratch_devs of=/dev/stdout 2022-09-20T20:06:16.154 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-09-20T20:06:16.155 DEBUG:teuthology.orchestra.run.smithi036:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-09-20T20:06:16.211 INFO:teuthology.orchestra.run.smithi036.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-09-20T20:06:16.211 INFO:teuthology.orchestra.run.smithi036.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-20T20:06:16.212 INFO:teuthology.orchestra.run.smithi036.stdout:Device: 6h/6d Inode: 356942 Links: 1 2022-09-20T20:06:16.212 INFO:teuthology.orchestra.run.smithi036.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-20T20:06:16.212 INFO:teuthology.orchestra.run.smithi036.stdout:Context: system_u:object_r:device_t:s0 2022-09-20T20:06:16.212 INFO:teuthology.orchestra.run.smithi036.stdout:Access: 2022-09-20 20:06:11.359153832 +0000 2022-09-20T20:06:16.212 INFO:teuthology.orchestra.run.smithi036.stdout:Modify: 2022-09-20 20:03:27.175343796 +0000 2022-09-20T20:06:16.213 INFO:teuthology.orchestra.run.smithi036.stdout:Change: 2022-09-20 20:03:27.175343796 +0000 2022-09-20T20:06:16.213 INFO:teuthology.orchestra.run.smithi036.stdout: Birth: - 2022-09-20T20:06:16.221 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records in 2022-09-20T20:06:16.222 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records out 2022-09-20T20:06:16.222 INFO:teuthology.orchestra.run.smithi036.stderr:512 bytes copied, 0.000225189 s, 2.3 MB/s 2022-09-20T20:06:16.224 DEBUG:teuthology.orchestra.run.smithi036:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-09-20T20:06:16.282 INFO:teuthology.orchestra.run.smithi036.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-09-20T20:06:16.282 INFO:teuthology.orchestra.run.smithi036.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-20T20:06:16.282 INFO:teuthology.orchestra.run.smithi036.stdout:Device: 6h/6d Inode: 363545 Links: 1 2022-09-20T20:06:16.282 INFO:teuthology.orchestra.run.smithi036.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-20T20:06:16.283 INFO:teuthology.orchestra.run.smithi036.stdout:Context: system_u:object_r:device_t:s0 2022-09-20T20:06:16.283 INFO:teuthology.orchestra.run.smithi036.stdout:Access: 2022-09-20 20:06:11.359153832 +0000 2022-09-20T20:06:16.283 INFO:teuthology.orchestra.run.smithi036.stdout:Modify: 2022-09-20 20:03:27.718338518 +0000 2022-09-20T20:06:16.283 INFO:teuthology.orchestra.run.smithi036.stdout:Change: 2022-09-20 20:03:27.718338518 +0000 2022-09-20T20:06:16.284 INFO:teuthology.orchestra.run.smithi036.stdout: Birth: - 2022-09-20T20:06:16.292 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records in 2022-09-20T20:06:16.292 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records out 2022-09-20T20:06:16.293 INFO:teuthology.orchestra.run.smithi036.stderr:512 bytes copied, 0.000209638 s, 2.4 MB/s 2022-09-20T20:06:16.295 DEBUG:teuthology.orchestra.run.smithi036:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-09-20T20:06:16.355 INFO:teuthology.orchestra.run.smithi036.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-09-20T20:06:16.356 INFO:teuthology.orchestra.run.smithi036.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-20T20:06:16.356 INFO:teuthology.orchestra.run.smithi036.stdout:Device: 6h/6d Inode: 360710 Links: 1 2022-09-20T20:06:16.356 INFO:teuthology.orchestra.run.smithi036.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-20T20:06:16.356 INFO:teuthology.orchestra.run.smithi036.stdout:Context: system_u:object_r:device_t:s0 2022-09-20T20:06:16.357 INFO:teuthology.orchestra.run.smithi036.stdout:Access: 2022-09-20 20:06:11.359153832 +0000 2022-09-20T20:06:16.357 INFO:teuthology.orchestra.run.smithi036.stdout:Modify: 2022-09-20 20:03:28.242333425 +0000 2022-09-20T20:06:16.357 INFO:teuthology.orchestra.run.smithi036.stdout:Change: 2022-09-20 20:03:28.242333425 +0000 2022-09-20T20:06:16.357 INFO:teuthology.orchestra.run.smithi036.stdout: Birth: - 2022-09-20T20:06:16.365 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records in 2022-09-20T20:06:16.366 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records out 2022-09-20T20:06:16.366 INFO:teuthology.orchestra.run.smithi036.stderr:512 bytes copied, 0.000186248 s, 2.7 MB/s 2022-09-20T20:06:16.368 DEBUG:teuthology.orchestra.run.smithi036:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-09-20T20:06:16.428 INFO:teuthology.orchestra.run.smithi036.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-09-20T20:06:16.428 INFO:teuthology.orchestra.run.smithi036.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-20T20:06:16.429 INFO:teuthology.orchestra.run.smithi036.stdout:Device: 6h/6d Inode: 363837 Links: 1 2022-09-20T20:06:16.429 INFO:teuthology.orchestra.run.smithi036.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-20T20:06:16.429 INFO:teuthology.orchestra.run.smithi036.stdout:Context: system_u:object_r:device_t:s0 2022-09-20T20:06:16.430 INFO:teuthology.orchestra.run.smithi036.stdout:Access: 2022-09-20 20:06:11.359153832 +0000 2022-09-20T20:06:16.430 INFO:teuthology.orchestra.run.smithi036.stdout:Modify: 2022-09-20 20:03:28.783328166 +0000 2022-09-20T20:06:16.430 INFO:teuthology.orchestra.run.smithi036.stdout:Change: 2022-09-20 20:03:28.783328166 +0000 2022-09-20T20:06:16.431 INFO:teuthology.orchestra.run.smithi036.stdout: Birth: - 2022-09-20T20:06:16.438 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records in 2022-09-20T20:06:16.439 INFO:teuthology.orchestra.run.smithi036.stderr:1+0 records out 2022-09-20T20:06:16.439 INFO:teuthology.orchestra.run.smithi036.stderr:512 bytes copied, 0.000211157 s, 2.4 MB/s 2022-09-20T20:06:16.441 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-09-20T20:06:16.441 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:16.442 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2022-09-20T20:06:16.461 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-09-20T20:06:16.461 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-09-20T20:06:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-09-20T20:06:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-20T20:06:16.519 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 358782 Links: 1 2022-09-20T20:06:16.520 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-20T20:06:16.520 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2022-09-20T20:06:16.520 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2022-09-20 20:06:11.620864909 +0000 2022-09-20T20:06:16.520 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2022-09-20 20:03:11.284827889 +0000 2022-09-20T20:06:16.521 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2022-09-20 20:03:11.284827889 +0000 2022-09-20T20:06:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2022-09-20T20:06:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2022-09-20T20:06:16.530 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2022-09-20T20:06:16.530 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000208186 s, 2.5 MB/s 2022-09-20T20:06:16.532 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-09-20T20:06:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-09-20T20:06:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-20T20:06:16.591 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 358849 Links: 1 2022-09-20T20:06:16.591 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-20T20:06:16.591 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2022-09-20T20:06:16.591 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2022-09-20 20:06:11.620864909 +0000 2022-09-20T20:06:16.591 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2022-09-20 20:03:11.824818047 +0000 2022-09-20T20:06:16.592 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2022-09-20 20:03:11.824818047 +0000 2022-09-20T20:06:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2022-09-20T20:06:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2022-09-20T20:06:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2022-09-20T20:06:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000235631 s, 2.2 MB/s 2022-09-20T20:06:16.603 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-09-20T20:06:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-09-20T20:06:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-20T20:06:16.661 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 362077 Links: 1 2022-09-20T20:06:16.661 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-20T20:06:16.661 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2022-09-20T20:06:16.661 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2022-09-20 20:06:11.620864909 +0000 2022-09-20T20:06:16.661 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2022-09-20 20:03:12.344808570 +0000 2022-09-20T20:06:16.662 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2022-09-20 20:03:12.344808570 +0000 2022-09-20T20:06:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2022-09-20T20:06:16.670 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2022-09-20T20:06:16.670 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2022-09-20T20:06:16.671 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000199889 s, 2.6 MB/s 2022-09-20T20:06:16.673 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-09-20T20:06:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-09-20T20:06:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-20T20:06:16.731 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 356181 Links: 1 2022-09-20T20:06:16.731 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-20T20:06:16.731 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2022-09-20T20:06:16.732 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2022-09-20 20:06:11.620864909 +0000 2022-09-20T20:06:16.732 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2022-09-20 20:03:13.077795211 +0000 2022-09-20T20:06:16.732 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2022-09-20 20:03:13.077795211 +0000 2022-09-20T20:06:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2022-09-20T20:06:16.740 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2022-09-20T20:06:16.740 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2022-09-20T20:06:16.741 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000289281 s, 1.8 MB/s 2022-09-20T20:06:16.743 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-09-20T20:06:16.744 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi036.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-09-20T20:06:16.744 INFO:tasks.ceph:Generating config... 2022-09-20T20:06:16.746 INFO:tasks.ceph:[client] client mount timeout = 600 2022-09-20T20:06:16.746 INFO:tasks.ceph:[client] debug client = 20 2022-09-20T20:06:16.747 INFO:tasks.ceph:[client] debug ms = 1 2022-09-20T20:06:16.747 INFO:tasks.ceph:[client] pid file = 2022-09-20T20:06:16.747 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-09-20T20:06:16.748 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-09-20T20:06:16.748 INFO:tasks.ceph:[global] ms die on skipped message = False 2022-09-20T20:06:16.748 INFO:tasks.ceph:[mds] debug mds = 20 2022-09-20T20:06:16.749 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-09-20T20:06:16.749 INFO:tasks.ceph:[mds] debug ms = 1 2022-09-20T20:06:16.749 INFO:tasks.ceph:[mds] mds debug frag = True 2022-09-20T20:06:16.750 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-09-20T20:06:16.750 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-09-20T20:06:16.750 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-09-20T20:06:16.751 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-09-20T20:06:16.751 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-09-20T20:06:16.751 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-09-20T20:06:16.752 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-09-20T20:06:16.752 INFO:tasks.ceph:[mgr] debug ms = 1 2022-09-20T20:06:16.752 INFO:tasks.ceph:[mon] debug mon = 20 2022-09-20T20:06:16.753 INFO:tasks.ceph:[mon] debug ms = 1 2022-09-20T20:06:16.753 INFO:tasks.ceph:[mon] debug paxos = 20 2022-09-20T20:06:16.753 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-09-20T20:06:16.754 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-09-20T20:06:16.754 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-09-20T20:06:16.754 INFO:tasks.ceph:[osd] debug bluefs = 20 2022-09-20T20:06:16.755 INFO:tasks.ceph:[osd] debug bluestore = 20 2022-09-20T20:06:16.755 INFO:tasks.ceph:[osd] debug ms = 1 2022-09-20T20:06:16.755 INFO:tasks.ceph:[osd] debug osd = 20 2022-09-20T20:06:16.755 INFO:tasks.ceph:[osd] debug rocksdb = 10 2022-09-20T20:06:16.756 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-09-20T20:06:16.756 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-09-20T20:06:16.756 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-09-20T20:06:16.757 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-09-20T20:06:16.757 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-09-20T20:06:16.757 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-09-20T20:06:16.758 INFO:tasks.ceph:Setting up mon.a... 2022-09-20T20:06:16.758 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-09-20T20:06:16.810 INFO:teuthology.orchestra.run.smithi036.stdout:creating /etc/ceph/ceph.keyring 2022-09-20T20:06:16.813 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-09-20T20:06:16.860 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-09-20T20:06:16.887 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.36'), ('mon.b', '172.21.15.115'), ('mon.c', '[v2:172.21.15.115:3301,v1:172.21.15.115:6790]')] 2022-09-20T20:06:16.888 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.36,172.21.15.115,[v2:172.21.15.115:3301,v1:172.21.15.115:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-09-20T20:06:16.889 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:16.889 DEBUG:teuthology.orchestra.run.smithi036:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-09-20T20:06:16.944 DEBUG:teuthology.orchestra.run.smithi036:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.36 --add b 172.21.15.115 --addv c '[v2:172.21.15.115:3301,v1:172.21.15.115:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:17.033 INFO:teuthology.orchestra.run.smithi036.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:17.034 INFO:teuthology.orchestra.run.smithi036.stdout:monmaptool: generated fsid 3c118203-7ef2-4e20-8eae-6178b988d685 2022-09-20T20:06:17.034 INFO:teuthology.orchestra.run.smithi036.stdout:setting min_mon_release = octopus 2022-09-20T20:06:17.035 INFO:teuthology.orchestra.run.smithi036.stdout:epoch 0 2022-09-20T20:06:17.035 INFO:teuthology.orchestra.run.smithi036.stdout:fsid 3c118203-7ef2-4e20-8eae-6178b988d685 2022-09-20T20:06:17.035 INFO:teuthology.orchestra.run.smithi036.stdout:last_changed 2022-09-20T20:06:17.033287+0000 2022-09-20T20:06:17.035 INFO:teuthology.orchestra.run.smithi036.stdout:created 2022-09-20T20:06:17.033287+0000 2022-09-20T20:06:17.036 INFO:teuthology.orchestra.run.smithi036.stdout:min_mon_release 15 (octopus) 2022-09-20T20:06:17.036 INFO:teuthology.orchestra.run.smithi036.stdout:election_strategy: 1 2022-09-20T20:06:17.036 INFO:teuthology.orchestra.run.smithi036.stdout:0: [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] mon.a 2022-09-20T20:06:17.036 INFO:teuthology.orchestra.run.smithi036.stdout:1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2022-09-20T20:06:17.036 INFO:teuthology.orchestra.run.smithi036.stdout:2: [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] mon.c 2022-09-20T20:06:17.037 INFO:teuthology.orchestra.run.smithi036.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-09-20T20:06:17.038 DEBUG:teuthology.orchestra.run.smithi036:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-09-20T20:06:17.055 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3c118203-7ef2-4e20-8eae-6178b988d685... 2022-09-20T20:06:17.056 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-09-20T20:06:17.098 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-09-20T20:06:17.142 INFO:teuthology.orchestra.run.smithi036.stdout:[global] 2022-09-20T20:06:17.142 INFO:teuthology.orchestra.run.smithi036.stdout: chdir = "" 2022-09-20T20:06:17.142 INFO:teuthology.orchestra.run.smithi036.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-09-20T20:06:17.143 INFO:teuthology.orchestra.run.smithi036.stdout: auth supported = cephx 2022-09-20T20:06:17.143 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.143 INFO:teuthology.orchestra.run.smithi036.stdout: filestore xattr use omap = true 2022-09-20T20:06:17.143 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.144 INFO:teuthology.orchestra.run.smithi036.stdout: mon clock drift allowed = 1.000 2022-09-20T20:06:17.144 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.144 INFO:teuthology.orchestra.run.smithi036.stdout: osd crush chooseleaf type = 0 2022-09-20T20:06:17.144 INFO:teuthology.orchestra.run.smithi036.stdout: auth debug = true 2022-09-20T20:06:17.144 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.145 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on old message = true 2022-09-20T20:06:17.145 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on bug = true 2022-09-20T20:06:17.145 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.145 INFO:teuthology.orchestra.run.smithi036.stdout: mon max pg per osd = 10000 # >= luminous 2022-09-20T20:06:17.146 INFO:teuthology.orchestra.run.smithi036.stdout: mon pg warn max object skew = 0 2022-09-20T20:06:17.146 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.146 INFO:teuthology.orchestra.run.smithi036.stdout: # disable pg_autoscaler by default for new pools 2022-09-20T20:06:17.146 INFO:teuthology.orchestra.run.smithi036.stdout: osd_pool_default_pg_autoscale_mode = off 2022-09-20T20:06:17.146 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.147 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default size = 2 2022-09-20T20:06:17.147 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.147 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow primary affinity = true 2022-09-20T20:06:17.147 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd allow pg remap = true 2022-09-20T20:06:17.148 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on legacy crush tunables = false 2022-09-20T20:06:17.148 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on crush straw calc version zero = false 2022-09-20T20:06:17.148 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on no sortbitwise = false 2022-09-20T20:06:17.148 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on osd down out interval zero = false 2022-09-20T20:06:17.149 INFO:teuthology.orchestra.run.smithi036.stdout: mon warn on too few osds = false 2022-09-20T20:06:17.149 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-09-20T20:06:17.149 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_pool_no_redundancy = false 2022-09-20T20:06:17.149 INFO:teuthology.orchestra.run.smithi036.stdout: mon_allow_pool_size_one = true 2022-09-20T20:06:17.150 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.150 INFO:teuthology.orchestra.run.smithi036.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-09-20T20:06:17.150 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.150 INFO:teuthology.orchestra.run.smithi036.stdout: osd default data pool replay window = 5 2022-09-20T20:06:17.150 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.151 INFO:teuthology.orchestra.run.smithi036.stdout: mon allow pool delete = true 2022-09-20T20:06:17.151 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.151 INFO:teuthology.orchestra.run.smithi036.stdout: mon cluster log file level = debug 2022-09-20T20:06:17.151 INFO:teuthology.orchestra.run.smithi036.stdout: debug asserts on shutdown = true 2022-09-20T20:06:17.152 INFO:teuthology.orchestra.run.smithi036.stdout: mon health detail to clog = false 2022-09-20T20:06:17.152 INFO:teuthology.orchestra.run.smithi036.stdout: mon host = "172.21.15.36,172.21.15.115,[v2:172.21.15.115:3301,v1:172.21.15.115:6790]" 2022-09-20T20:06:17.152 INFO:teuthology.orchestra.run.smithi036.stdout: ms die on skipped message = False 2022-09-20T20:06:17.152 INFO:teuthology.orchestra.run.smithi036.stdout: fsid = 3c118203-7ef2-4e20-8eae-6178b988d685 2022-09-20T20:06:17.152 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.153 INFO:teuthology.orchestra.run.smithi036.stdout:[osd] 2022-09-20T20:06:17.153 INFO:teuthology.orchestra.run.smithi036.stdout: osd journal size = 100 2022-09-20T20:06:17.153 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.153 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub load threshold = 5.0 2022-09-20T20:06:17.154 INFO:teuthology.orchestra.run.smithi036.stdout: osd scrub max interval = 600 2022-09-20T20:06:17.154 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.154 INFO:teuthology.orchestra.run.smithi036.stdout: osd recover clone overlap = true 2022-09-20T20:06:17.154 INFO:teuthology.orchestra.run.smithi036.stdout: osd recovery max chunk = 1048576 2022-09-20T20:06:17.155 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.155 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug shutdown = true 2022-09-20T20:06:17.155 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug op order = true 2022-09-20T20:06:17.155 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug verify stray on activate = true 2022-09-20T20:06:17.155 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.156 INFO:teuthology.orchestra.run.smithi036.stdout: osd open classes on start = true 2022-09-20T20:06:17.156 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug pg log writeout = true 2022-09-20T20:06:17.156 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.156 INFO:teuthology.orchestra.run.smithi036.stdout: osd deep scrub update digest min age = 30 2022-09-20T20:06:17.157 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.157 INFO:teuthology.orchestra.run.smithi036.stdout: osd map max advance = 10 2022-09-20T20:06:17.157 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.157 INFO:teuthology.orchestra.run.smithi036.stdout: journal zero on create = true 2022-09-20T20:06:17.158 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.158 INFO:teuthology.orchestra.run.smithi036.stdout: filestore ondisk finisher threads = 3 2022-09-20T20:06:17.158 INFO:teuthology.orchestra.run.smithi036.stdout: filestore apply finisher threads = 3 2022-09-20T20:06:17.158 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.159 INFO:teuthology.orchestra.run.smithi036.stdout: bdev debug aio = true 2022-09-20T20:06:17.159 INFO:teuthology.orchestra.run.smithi036.stdout: osd debug misdirected ops = true 2022-09-20T20:06:17.159 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore block size = 96636764160 2022-09-20T20:06:17.159 INFO:teuthology.orchestra.run.smithi036.stdout: bluestore fsck on mount = True 2022-09-20T20:06:17.159 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluefs = 20 2022-09-20T20:06:17.160 INFO:teuthology.orchestra.run.smithi036.stdout: debug bluestore = 20 2022-09-20T20:06:17.160 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-09-20T20:06:17.160 INFO:teuthology.orchestra.run.smithi036.stdout: debug osd = 20 2022-09-20T20:06:17.160 INFO:teuthology.orchestra.run.smithi036.stdout: debug rocksdb = 10 2022-09-20T20:06:17.160 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd backfillfull_ratio = 0.85 2022-09-20T20:06:17.161 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd full ratio = 0.9 2022-09-20T20:06:17.161 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd nearfull ratio = 0.8 2022-09-20T20:06:17.161 INFO:teuthology.orchestra.run.smithi036.stdout: osd failsafe full ratio = 0.95 2022-09-20T20:06:17.161 INFO:teuthology.orchestra.run.smithi036.stdout: osd objectstore = bluestore 2022-09-20T20:06:17.162 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2022-09-20T20:06:17.162 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.162 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr] 2022-09-20T20:06:17.162 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-09-20T20:06:17.162 INFO:teuthology.orchestra.run.smithi036.stdout: debug mgr = 20 2022-09-20T20:06:17.163 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2022-09-20T20:06:17.163 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2022-09-20T20:06:17.163 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2022-09-20T20:06:17.163 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2022-09-20T20:06:17.163 INFO:teuthology.orchestra.run.smithi036.stdout: mgr/telemetry/nag = false 2022-09-20T20:06:17.164 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.164 INFO:teuthology.orchestra.run.smithi036.stdout:[mon] 2022-09-20T20:06:17.164 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-09-20T20:06:17.164 INFO:teuthology.orchestra.run.smithi036.stdout: debug mon = 20 2022-09-20T20:06:17.165 INFO:teuthology.orchestra.run.smithi036.stdout: debug paxos = 20 2022-09-20T20:06:17.165 INFO:teuthology.orchestra.run.smithi036.stdout: debug auth = 20 2022-09-20T20:06:17.165 INFO:teuthology.orchestra.run.smithi036.stdout: mon data avail warn = 5 2022-09-20T20:06:17.165 INFO:teuthology.orchestra.run.smithi036.stdout: mon mgr mkfs grace = 240 2022-09-20T20:06:17.166 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min pgs per osd = 4 2022-09-20T20:06:17.166 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd reporter subtree level = osd 2022-09-20T20:06:17.166 INFO:teuthology.orchestra.run.smithi036.stdout: mon osd prime pg temp = true 2022-09-20T20:06:17.167 INFO:teuthology.orchestra.run.smithi036.stdout: mon reweight min bytes per osd = 10 2022-09-20T20:06:17.167 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.167 INFO:teuthology.orchestra.run.smithi036.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-09-20T20:06:17.168 INFO:teuthology.orchestra.run.smithi036.stdout: auth mon ticket ttl = 660 # 11m 2022-09-20T20:06:17.168 INFO:teuthology.orchestra.run.smithi036.stdout: auth service ticket ttl = 240 # 4m 2022-09-20T20:06:17.169 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.169 INFO:teuthology.orchestra.run.smithi036.stdout: # don't complain about insecure global_id in the test suite 2022-09-20T20:06:17.170 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-09-20T20:06:17.170 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-09-20T20:06:17.171 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.171 INFO:teuthology.orchestra.run.smithi036.stdout: # 1m isn't quite enough 2022-09-20T20:06:17.172 INFO:teuthology.orchestra.run.smithi036.stdout: mon_down_mkfs_grace = 2m 2022-09-20T20:06:17.172 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.172 INFO:teuthology.orchestra.run.smithi036.stdout: mon_warn_on_filestore_osds = false 2022-09-20T20:06:17.173 INFO:teuthology.orchestra.run.smithi036.stdout: mon op complaint time = 120 2022-09-20T20:06:17.173 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:17.173 INFO:teuthology.orchestra.run.smithi036.stdout:[client] 2022-09-20T20:06:17.174 INFO:teuthology.orchestra.run.smithi036.stdout: rgw cache enabled = true 2022-09-20T20:06:17.174 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable ops log = true 2022-09-20T20:06:17.174 INFO:teuthology.orchestra.run.smithi036.stdout: rgw enable usage log = true 2022-09-20T20:06:17.175 INFO:teuthology.orchestra.run.smithi036.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-09-20T20:06:17.175 INFO:teuthology.orchestra.run.smithi036.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-09-20T20:06:17.175 INFO:teuthology.orchestra.run.smithi036.stdout: client mount timeout = 600 2022-09-20T20:06:17.175 INFO:teuthology.orchestra.run.smithi036.stdout: debug client = 20 2022-09-20T20:06:17.175 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-09-20T20:06:17.176 INFO:teuthology.orchestra.run.smithi036.stdout: pid file = "" 2022-09-20T20:06:17.176 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 900 2022-09-20T20:06:17.176 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 900 2022-09-20T20:06:17.176 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.a] 2022-09-20T20:06:17.176 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.b] 2022-09-20T20:06:17.177 INFO:teuthology.orchestra.run.smithi036.stdout:[mon.c] 2022-09-20T20:06:17.177 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.a] 2022-09-20T20:06:17.177 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.c] 2022-09-20T20:06:17.177 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.b] 2022-09-20T20:06:17.177 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.d] 2022-09-20T20:06:17.178 INFO:teuthology.orchestra.run.smithi036.stdout:[mds] 2022-09-20T20:06:17.178 INFO:teuthology.orchestra.run.smithi036.stdout: debug mds = 20 2022-09-20T20:06:17.178 INFO:teuthology.orchestra.run.smithi036.stdout: debug mds balancer = 20 2022-09-20T20:06:17.178 INFO:teuthology.orchestra.run.smithi036.stdout: debug ms = 1 2022-09-20T20:06:17.179 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug frag = True 2022-09-20T20:06:17.179 INFO:teuthology.orchestra.run.smithi036.stdout: mds debug scatterstat = True 2022-09-20T20:06:17.179 INFO:teuthology.orchestra.run.smithi036.stdout: mds op complaint time = 180 2022-09-20T20:06:17.179 INFO:teuthology.orchestra.run.smithi036.stdout: mds verify scatter = True 2022-09-20T20:06:17.180 INFO:teuthology.orchestra.run.smithi036.stdout: osd op complaint time = 180 2022-09-20T20:06:17.180 INFO:teuthology.orchestra.run.smithi036.stdout: rados mon op timeout = 900 2022-09-20T20:06:17.180 INFO:teuthology.orchestra.run.smithi036.stdout: rados osd op timeout = 900 2022-09-20T20:06:17.182 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2022-09-20T20:06:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: chdir = "" 2022-09-20T20:06:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-09-20T20:06:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: auth supported = cephx 2022-09-20T20:06:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: filestore xattr use omap = true 2022-09-20T20:06:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: mon clock drift allowed = 1.000 2022-09-20T20:06:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: osd crush chooseleaf type = 0 2022-09-20T20:06:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: auth debug = true 2022-09-20T20:06:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on old message = true 2022-09-20T20:06:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on bug = true 2022-09-20T20:06:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: mon max pg per osd = 10000 # >= luminous 2022-09-20T20:06:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: mon pg warn max object skew = 0 2022-09-20T20:06:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: # disable pg_autoscaler by default for new pools 2022-09-20T20:06:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: osd_pool_default_pg_autoscale_mode = off 2022-09-20T20:06:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default size = 2 2022-09-20T20:06:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow primary affinity = true 2022-09-20T20:06:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow pg remap = true 2022-09-20T20:06:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on legacy crush tunables = false 2022-09-20T20:06:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on crush straw calc version zero = false 2022-09-20T20:06:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on no sortbitwise = false 2022-09-20T20:06:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on osd down out interval zero = false 2022-09-20T20:06:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on too few osds = false 2022-09-20T20:06:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-09-20T20:06:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_no_redundancy = false 2022-09-20T20:06:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: mon_allow_pool_size_one = true 2022-09-20T20:06:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-09-20T20:06:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: osd default data pool replay window = 5 2022-09-20T20:06:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: mon allow pool delete = true 2022-09-20T20:06:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: mon cluster log file level = debug 2022-09-20T20:06:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: debug asserts on shutdown = true 2022-09-20T20:06:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: mon health detail to clog = false 2022-09-20T20:06:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: mon host = "172.21.15.36,172.21.15.115,[v2:172.21.15.115:3301,v1:172.21.15.115:6790]" 2022-09-20T20:06:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on skipped message = False 2022-09-20T20:06:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = 3c118203-7ef2-4e20-8eae-6178b988d685 2022-09-20T20:06:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.193 INFO:teuthology.orchestra.run.smithi115.stdout:[osd] 2022-09-20T20:06:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: osd journal size = 100 2022-09-20T20:06:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub load threshold = 5.0 2022-09-20T20:06:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub max interval = 600 2022-09-20T20:06:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: osd recover clone overlap = true 2022-09-20T20:06:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: osd recovery max chunk = 1048576 2022-09-20T20:06:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug shutdown = true 2022-09-20T20:06:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug op order = true 2022-09-20T20:06:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify stray on activate = true 2022-09-20T20:06:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: osd open classes on start = true 2022-09-20T20:06:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug pg log writeout = true 2022-09-20T20:06:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: osd deep scrub update digest min age = 30 2022-09-20T20:06:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: osd map max advance = 10 2022-09-20T20:06:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: journal zero on create = true 2022-09-20T20:06:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: filestore ondisk finisher threads = 3 2022-09-20T20:06:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: filestore apply finisher threads = 3 2022-09-20T20:06:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: bdev debug aio = true 2022-09-20T20:06:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug misdirected ops = true 2022-09-20T20:06:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore block size = 96636764160 2022-09-20T20:06:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore fsck on mount = True 2022-09-20T20:06:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluefs = 20 2022-09-20T20:06:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluestore = 20 2022-09-20T20:06:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2022-09-20T20:06:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: debug osd = 20 2022-09-20T20:06:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: debug rocksdb = 10 2022-09-20T20:06:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd backfillfull_ratio = 0.85 2022-09-20T20:06:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd full ratio = 0.9 2022-09-20T20:06:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd nearfull ratio = 0.8 2022-09-20T20:06:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: osd failsafe full ratio = 0.95 2022-09-20T20:06:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: osd objectstore = bluestore 2022-09-20T20:06:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: osd op complaint time = 180 2022-09-20T20:06:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.204 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr] 2022-09-20T20:06:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2022-09-20T20:06:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: debug mgr = 20 2022-09-20T20:06:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2022-09-20T20:06:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2022-09-20T20:06:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2022-09-20T20:06:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2022-09-20T20:06:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: mgr/telemetry/nag = false 2022-09-20T20:06:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.206 INFO:teuthology.orchestra.run.smithi115.stdout:[mon] 2022-09-20T20:06:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2022-09-20T20:06:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2022-09-20T20:06:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: debug paxos = 20 2022-09-20T20:06:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2022-09-20T20:06:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: mon data avail warn = 5 2022-09-20T20:06:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: mon mgr mkfs grace = 240 2022-09-20T20:06:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2022-09-20T20:06:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd reporter subtree level = osd 2022-09-20T20:06:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd prime pg temp = true 2022-09-20T20:06:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2022-09-20T20:06:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-09-20T20:06:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: auth mon ticket ttl = 660 # 11m 2022-09-20T20:06:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: auth service ticket ttl = 240 # 4m 2022-09-20T20:06:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: # don't complain about insecure global_id in the test suite 2022-09-20T20:06:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-09-20T20:06:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-09-20T20:06:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: # 1m isn't quite enough 2022-09-20T20:06:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: mon_down_mkfs_grace = 2m 2022-09-20T20:06:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_filestore_osds = false 2022-09-20T20:06:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: mon op complaint time = 120 2022-09-20T20:06:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: 2022-09-20T20:06:17.213 INFO:teuthology.orchestra.run.smithi115.stdout:[client] 2022-09-20T20:06:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: rgw cache enabled = true 2022-09-20T20:06:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable ops log = true 2022-09-20T20:06:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable usage log = true 2022-09-20T20:06:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-09-20T20:06:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-09-20T20:06:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: client mount timeout = 600 2022-09-20T20:06:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: debug client = 20 2022-09-20T20:06:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2022-09-20T20:06:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = "" 2022-09-20T20:06:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: rados mon op timeout = 900 2022-09-20T20:06:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: rados osd op timeout = 900 2022-09-20T20:06:17.216 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.a] 2022-09-20T20:06:17.216 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.b] 2022-09-20T20:06:17.216 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.c] 2022-09-20T20:06:17.216 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.a] 2022-09-20T20:06:17.217 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.c] 2022-09-20T20:06:17.217 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.b] 2022-09-20T20:06:17.217 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.d] 2022-09-20T20:06:17.217 INFO:teuthology.orchestra.run.smithi115.stdout:[mds] 2022-09-20T20:06:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: debug mds = 20 2022-09-20T20:06:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: debug mds balancer = 20 2022-09-20T20:06:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2022-09-20T20:06:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: mds debug frag = True 2022-09-20T20:06:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: mds debug scatterstat = True 2022-09-20T20:06:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: mds op complaint time = 180 2022-09-20T20:06:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: mds verify scatter = True 2022-09-20T20:06:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: osd op complaint time = 180 2022-09-20T20:06:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: rados mon op timeout = 900 2022-09-20T20:06:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: rados osd op timeout = 900 2022-09-20T20:06:17.220 INFO:tasks.ceph:Creating admin key on mon.a... 2022-09-20T20:06:17.220 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-09-20T20:06:17.279 INFO:tasks.ceph:Copying monmap to all nodes... 2022-09-20T20:06:17.280 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:17.280 DEBUG:teuthology.orchestra.run.smithi036:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-09-20T20:06:17.297 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:17.298 DEBUG:teuthology.orchestra.run.smithi036:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-09-20T20:06:17.354 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi036.front.sepia.ceph.com 2022-09-20T20:06:17.355 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:17.355 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd of=/etc/ceph/ceph.keyring 2022-09-20T20:06:17.355 DEBUG:teuthology.orchestra.run.smithi036:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-09-20T20:06:17.432 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:17.432 DEBUG:teuthology.orchestra.run.smithi036:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:17.489 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi115.front.sepia.ceph.com 2022-09-20T20:06:17.490 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:17.490 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.keyring 2022-09-20T20:06:17.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-09-20T20:06:17.529 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:17.529 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:17.586 INFO:tasks.ceph:Setting up mon nodes... 2022-09-20T20:06:17.587 INFO:tasks.ceph:Setting up mgr nodes... 2022-09-20T20:06:17.587 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-09-20T20:06:17.642 INFO:teuthology.orchestra.run.smithi036.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-09-20T20:06:17.644 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-09-20T20:06:17.708 INFO:teuthology.orchestra.run.smithi115.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-09-20T20:06:17.711 INFO:tasks.ceph:Setting up mds nodes... 2022-09-20T20:06:17.712 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-09-20T20:06:17.766 INFO:teuthology.orchestra.run.smithi036.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-09-20T20:06:17.768 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-09-20T20:06:17.796 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-09-20T20:06:17.891 INFO:teuthology.orchestra.run.smithi036.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-09-20T20:06:17.893 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-09-20T20:06:17.921 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-09-20T20:06:17.975 INFO:teuthology.orchestra.run.smithi115.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-09-20T20:06:17.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-09-20T20:06:18.004 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2022-09-20T20:06:18.097 INFO:teuthology.orchestra.run.smithi115.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2022-09-20T20:06:18.099 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2022-09-20T20:06:18.126 INFO:tasks.ceph_client:Setting up client nodes... 2022-09-20T20:06:18.126 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-09-20T20:06:18.171 INFO:teuthology.orchestra.run.smithi036.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-09-20T20:06:18.185 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2022-09-20T20:06:18.228 INFO:teuthology.orchestra.run.smithi036.stdout:creating /etc/ceph/ceph.client.2.keyring 2022-09-20T20:06:18.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-09-20T20:06:18.285 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-09-20T20:06:18.298 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2022-09-20T20:06:18.381 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.client.3.keyring 2022-09-20T20:06:18.394 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-09-20T20:06:18.394 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi036.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-09-20T20:06:18.395 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-09-20T20:06:18.423 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-09-20T20:06:18.423 INFO:tasks.ceph:role: osd.0 2022-09-20T20:06:18.424 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi036.front.sepia.ceph.com 2022-09-20T20:06:18.424 DEBUG:teuthology.orchestra.run.smithi036:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-09-20T20:06:18.498 INFO:teuthology.orchestra.run.smithi036.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-09-20T20:06:18.498 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-20T20:06:18.499 INFO:teuthology.orchestra.run.smithi036.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-20T20:06:18.499 INFO:teuthology.orchestra.run.smithi036.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-20T20:06:18.499 INFO:teuthology.orchestra.run.smithi036.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-20T20:06:18.499 INFO:teuthology.orchestra.run.smithi036.stdout: = sunit=0 swidth=0 blks 2022-09-20T20:06:18.499 INFO:teuthology.orchestra.run.smithi036.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-20T20:06:18.500 INFO:teuthology.orchestra.run.smithi036.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-20T20:06:18.500 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-20T20:06:18.500 INFO:teuthology.orchestra.run.smithi036.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-20T20:06:18.931 INFO:teuthology.orchestra.run.smithi036.stdout:Discarding blocks...Done. 2022-09-20T20:06:18.932 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi036.front.sepia.ceph.com -o noatime 2022-09-20T20:06:18.933 DEBUG:teuthology.orchestra.run.smithi036:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-09-20T20:06:19.083 DEBUG:teuthology.orchestra.run.smithi036:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-09-20T20:06:19.118 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-09-20T20:06:19.187 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-09-20T20:06:19.188 INFO:tasks.ceph:role: osd.1 2022-09-20T20:06:19.188 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi036.front.sepia.ceph.com 2022-09-20T20:06:19.188 DEBUG:teuthology.orchestra.run.smithi036:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-09-20T20:06:19.262 INFO:teuthology.orchestra.run.smithi036.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-09-20T20:06:19.263 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-20T20:06:19.263 INFO:teuthology.orchestra.run.smithi036.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-20T20:06:19.263 INFO:teuthology.orchestra.run.smithi036.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-20T20:06:19.263 INFO:teuthology.orchestra.run.smithi036.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-20T20:06:19.264 INFO:teuthology.orchestra.run.smithi036.stdout: = sunit=0 swidth=0 blks 2022-09-20T20:06:19.264 INFO:teuthology.orchestra.run.smithi036.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-20T20:06:19.264 INFO:teuthology.orchestra.run.smithi036.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-20T20:06:19.264 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-20T20:06:19.265 INFO:teuthology.orchestra.run.smithi036.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-20T20:06:19.686 INFO:teuthology.orchestra.run.smithi036.stdout:Discarding blocks...Done. 2022-09-20T20:06:19.688 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi036.front.sepia.ceph.com -o noatime 2022-09-20T20:06:19.689 DEBUG:teuthology.orchestra.run.smithi036:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-09-20T20:06:19.796 DEBUG:teuthology.orchestra.run.smithi036:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-09-20T20:06:19.830 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-09-20T20:06:19.899 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-09-20T20:06:19.899 INFO:tasks.ceph:role: osd.2 2022-09-20T20:06:19.900 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi036.front.sepia.ceph.com 2022-09-20T20:06:19.900 DEBUG:teuthology.orchestra.run.smithi036:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-09-20T20:06:19.975 INFO:teuthology.orchestra.run.smithi036.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-09-20T20:06:19.976 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-20T20:06:19.976 INFO:teuthology.orchestra.run.smithi036.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-20T20:06:19.976 INFO:teuthology.orchestra.run.smithi036.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-20T20:06:19.976 INFO:teuthology.orchestra.run.smithi036.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-20T20:06:19.977 INFO:teuthology.orchestra.run.smithi036.stdout: = sunit=0 swidth=0 blks 2022-09-20T20:06:19.977 INFO:teuthology.orchestra.run.smithi036.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-20T20:06:19.977 INFO:teuthology.orchestra.run.smithi036.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-20T20:06:19.977 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-20T20:06:19.977 INFO:teuthology.orchestra.run.smithi036.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-20T20:06:20.421 INFO:teuthology.orchestra.run.smithi036.stdout:Discarding blocks...Done. 2022-09-20T20:06:20.424 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi036.front.sepia.ceph.com -o noatime 2022-09-20T20:06:20.424 DEBUG:teuthology.orchestra.run.smithi036:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-09-20T20:06:20.581 DEBUG:teuthology.orchestra.run.smithi036:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-09-20T20:06:20.617 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-09-20T20:06:20.688 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-09-20T20:06:20.689 INFO:tasks.ceph:role: osd.3 2022-09-20T20:06:20.690 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi036.front.sepia.ceph.com 2022-09-20T20:06:20.690 DEBUG:teuthology.orchestra.run.smithi036:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-09-20T20:06:20.765 INFO:teuthology.orchestra.run.smithi036.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-09-20T20:06:20.766 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-20T20:06:20.766 INFO:teuthology.orchestra.run.smithi036.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-20T20:06:20.767 INFO:teuthology.orchestra.run.smithi036.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-20T20:06:20.767 INFO:teuthology.orchestra.run.smithi036.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-20T20:06:20.768 INFO:teuthology.orchestra.run.smithi036.stdout: = sunit=0 swidth=0 blks 2022-09-20T20:06:20.768 INFO:teuthology.orchestra.run.smithi036.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-20T20:06:20.769 INFO:teuthology.orchestra.run.smithi036.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-20T20:06:20.769 INFO:teuthology.orchestra.run.smithi036.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-20T20:06:20.769 INFO:teuthology.orchestra.run.smithi036.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-20T20:06:21.211 INFO:teuthology.orchestra.run.smithi036.stdout:Discarding blocks...Done. 2022-09-20T20:06:21.213 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi036.front.sepia.ceph.com -o noatime 2022-09-20T20:06:21.213 DEBUG:teuthology.orchestra.run.smithi036:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-09-20T20:06:21.388 DEBUG:teuthology.orchestra.run.smithi036:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-09-20T20:06:21.425 DEBUG:teuthology.orchestra.run.smithi036:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:21.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:21.519+0000 7f61620445c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-09-20T20:06:21.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:21.519+0000 7f61620445c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-09-20T20:06:21.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:21.519+0000 7f61620445c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-09-20T20:06:21.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:21.519+0000 7f61620445c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-09-20T20:06:21.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:21.519+0000 7f61620445c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-09-20T20:06:21.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:21.520+0000 7f61620445c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-09-20T20:06:24.374 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-09-20T20:06:24.407 DEBUG:teuthology.orchestra.run.smithi036:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:24.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:24.499+0000 7ffae0c6c5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-09-20T20:06:24.501 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:24.499+0000 7ffae0c6c5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-09-20T20:06:24.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:24.499+0000 7ffae0c6c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-09-20T20:06:24.502 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:24.499+0000 7ffae0c6c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-09-20T20:06:24.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:24.499+0000 7ffae0c6c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-09-20T20:06:24.503 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:24.500+0000 7ffae0c6c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-09-20T20:06:27.362 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-09-20T20:06:27.396 DEBUG:teuthology.orchestra.run.smithi036:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:27.491 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:27.488+0000 7f448e37d5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-09-20T20:06:27.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:27.488+0000 7f448e37d5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-09-20T20:06:27.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:27.488+0000 7f448e37d5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-09-20T20:06:27.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:27.488+0000 7f448e37d5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-09-20T20:06:27.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:27.488+0000 7f448e37d5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-09-20T20:06:27.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:27.489+0000 7f448e37d5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-09-20T20:06:30.353 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-09-20T20:06:30.386 DEBUG:teuthology.orchestra.run.smithi036:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:30.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:30.477+0000 7fc0446d55c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-09-20T20:06:30.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:30.478+0000 7fc0446d55c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-09-20T20:06:30.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:30.478+0000 7fc0446d55c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-09-20T20:06:30.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:30.478+0000 7fc0446d55c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-09-20T20:06:30.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:30.478+0000 7fc0446d55c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-09-20T20:06:30.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:30.478+0000 7fc0446d55c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-09-20T20:06:33.346 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-09-20T20:06:33.379 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-09-20T20:06:33.413 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-09-20T20:06:33.414 INFO:tasks.ceph:role: osd.4 2022-09-20T20:06:33.414 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com 2022-09-20T20:06:33.414 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-09-20T20:06:33.491 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-09-20T20:06:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-20T20:06:33.491 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-20T20:06:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-20T20:06:33.492 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-20T20:06:33.492 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2022-09-20T20:06:33.493 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-20T20:06:33.493 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-20T20:06:33.493 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-20T20:06:33.493 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-20T20:06:33.916 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2022-09-20T20:06:33.918 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2022-09-20T20:06:33.918 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-09-20T20:06:34.093 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-09-20T20:06:34.133 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-09-20T20:06:34.203 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-09-20T20:06:34.204 INFO:tasks.ceph:role: osd.5 2022-09-20T20:06:34.204 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com 2022-09-20T20:06:34.207 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-09-20T20:06:34.279 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-09-20T20:06:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-20T20:06:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-20T20:06:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-20T20:06:34.281 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-20T20:06:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2022-09-20T20:06:34.282 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-20T20:06:34.283 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-20T20:06:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-20T20:06:34.284 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-20T20:06:34.702 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2022-09-20T20:06:34.704 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2022-09-20T20:06:34.704 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-09-20T20:06:34.823 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-09-20T20:06:34.857 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-09-20T20:06:34.926 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-09-20T20:06:34.926 INFO:tasks.ceph:role: osd.6 2022-09-20T20:06:34.927 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com 2022-09-20T20:06:34.927 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-09-20T20:06:35.001 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-09-20T20:06:35.002 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-20T20:06:35.002 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-20T20:06:35.002 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-20T20:06:35.003 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-20T20:06:35.003 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2022-09-20T20:06:35.003 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-20T20:06:35.003 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-20T20:06:35.004 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-20T20:06:35.004 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-20T20:06:35.455 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2022-09-20T20:06:35.456 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2022-09-20T20:06:35.457 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-09-20T20:06:35.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-09-20T20:06:35.635 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-09-20T20:06:35.704 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-09-20T20:06:35.704 INFO:tasks.ceph:role: osd.7 2022-09-20T20:06:35.705 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com 2022-09-20T20:06:35.705 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-09-20T20:06:35.779 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-09-20T20:06:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-20T20:06:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-20T20:06:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-20T20:06:35.781 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-20T20:06:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2022-09-20T20:06:35.782 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-20T20:06:35.782 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-20T20:06:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-20T20:06:35.783 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-20T20:06:36.224 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2022-09-20T20:06:36.226 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2022-09-20T20:06:36.226 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-09-20T20:06:36.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-09-20T20:06:36.450 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:36.542 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:36.540+0000 7f13895545c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-09-20T20:06:36.543 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:36.540+0000 7f13895545c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-09-20T20:06:36.543 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:36.540+0000 7f13895545c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-09-20T20:06:36.543 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:36.540+0000 7f13895545c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-09-20T20:06:36.543 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:36.540+0000 7f13895545c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-09-20T20:06:36.544 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:36.541+0000 7f13895545c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-09-20T20:06:39.400 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-09-20T20:06:39.434 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:39.525 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:39.522+0000 7f2dccb7a5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-09-20T20:06:39.526 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:39.522+0000 7f2dccb7a5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-09-20T20:06:39.526 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:39.522+0000 7f2dccb7a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-09-20T20:06:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:39.522+0000 7f2dccb7a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-09-20T20:06:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:39.522+0000 7f2dccb7a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-09-20T20:06:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:39.523+0000 7f2dccb7a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-09-20T20:06:42.389 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-09-20T20:06:42.421 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:42.512 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:42.510+0000 7f21f65a25c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-09-20T20:06:42.513 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:42.510+0000 7f21f65a25c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-09-20T20:06:42.513 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:42.510+0000 7f21f65a25c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-09-20T20:06:42.513 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:42.510+0000 7f21f65a25c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-09-20T20:06:42.513 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:42.510+0000 7f21f65a25c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-09-20T20:06:42.513 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:42.511+0000 7f21f65a25c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-09-20T20:06:45.372 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-09-20T20:06:45.402 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:45.490 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:45.488+0000 7f0a4df925c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-09-20T20:06:45.491 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:45.488+0000 7f0a4df925c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-09-20T20:06:45.491 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:45.488+0000 7f0a4df925c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-09-20T20:06:45.491 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:45.488+0000 7f0a4df925c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-09-20T20:06:45.492 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:45.488+0000 7f0a4df925c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-09-20T20:06:45.492 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:45.489+0000 7f0a4df925c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-09-20T20:06:46.013 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-09-20T20:06:46.017 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-09-20T20:06:48.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-09-20T20:06:48.373 INFO:tasks.ceph:Reading keys from all nodes... 2022-09-20T20:06:48.375 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:48.375 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-09-20T20:06:48.408 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:48.409 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-09-20T20:06:48.479 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:48.480 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-09-20T20:06:48.550 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:48.550 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-09-20T20:06:48.620 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:48.621 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-09-20T20:06:48.690 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:48.691 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-09-20T20:06:48.760 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:48.761 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-09-20T20:06:48.830 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:48.830 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-09-20T20:06:48.862 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:48.863 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-09-20T20:06:48.932 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:48.933 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2022-09-20T20:06:49.002 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:49.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2022-09-20T20:06:49.070 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:49.070 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2022-09-20T20:06:49.139 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:49.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2022-09-20T20:06:49.206 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:49.207 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2022-09-20T20:06:49.273 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:49.274 DEBUG:teuthology.orchestra.run.smithi036:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-09-20T20:06:49.295 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:49.296 DEBUG:teuthology.orchestra.run.smithi036:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2022-09-20T20:06:49.355 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:49.356 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-09-20T20:06:49.374 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:49.375 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2022-09-20T20:06:49.432 INFO:tasks.ceph:Adding keys to all mons... 2022-09-20T20:06:49.433 DEBUG:teuthology.orchestra.run.smithi036:> sudo tee -a /etc/ceph/ceph.keyring 2022-09-20T20:06:49.436 DEBUG:teuthology.orchestra.run.smithi115:> sudo tee -a /etc/ceph/ceph.keyring 2022-09-20T20:06:49.476 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr.y] 2022-09-20T20:06:49.476 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA5HSpjKyQ/JhAAUr0WiEf3Ir6byeSc/VMU7A== 2022-09-20T20:06:49.476 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.a] 2022-09-20T20:06:49.476 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA5HSpjj7unLRAAiVEe8hkQ+5w5pp58yzahzA== 2022-09-20T20:06:49.477 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.c] 2022-09-20T20:06:49.477 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA5HSpjDc4YNRAAFPEP0OI1c7F3BBozU153yw== 2022-09-20T20:06:49.477 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.0] 2022-09-20T20:06:49.477 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA9HSpjtXsGHxAAHjD2WP7PAVHK7xKkniE+HQ== 2022-09-20T20:06:49.478 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.1] 2022-09-20T20:06:49.478 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQBAHSpjUj7FHRAAnwKmrJmrZMBcMBxuwiBNTw== 2022-09-20T20:06:49.478 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.2] 2022-09-20T20:06:49.478 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQBDHSpjVy0vHRAAYiuPX+QHWTGmVXnFGCC78g== 2022-09-20T20:06:49.479 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.3] 2022-09-20T20:06:49.479 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQBGHSpjOkuLHBAAp/eVepQ7mpC4ArvTA28lnQ== 2022-09-20T20:06:49.479 INFO:teuthology.orchestra.run.smithi036.stdout:[mgr.x] 2022-09-20T20:06:49.479 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA5HSpjx6E1KhAAwSo6SMtmdCuyay9YaokOGQ== 2022-09-20T20:06:49.479 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.b] 2022-09-20T20:06:49.480 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA5HSpj8vUVOhAAkT7gRGCPj8NLuXqjSorJUw== 2022-09-20T20:06:49.480 INFO:teuthology.orchestra.run.smithi036.stdout:[mds.d] 2022-09-20T20:06:49.480 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA6HSpjZg7HBRAA3Os/GI42y1xP9Grdis31LA== 2022-09-20T20:06:49.480 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.4] 2022-09-20T20:06:49.480 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQBMHSpj62pHIBAA6MUdq1cGl4ohg1Xwh6pD9g== 2022-09-20T20:06:49.481 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.5] 2022-09-20T20:06:49.481 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQBPHSpjh2YyHxAAnEZj9ld68uVOw2u5/3SSIw== 2022-09-20T20:06:49.481 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.6] 2022-09-20T20:06:49.481 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQBSHSpjq4p9HhAA6IHnvrEAN9PGvHtS2AsbLA== 2022-09-20T20:06:49.481 INFO:teuthology.orchestra.run.smithi036.stdout:[osd.7] 2022-09-20T20:06:49.482 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQBVHSpjG2wvHRAAjyjEC2nxxLy0EkgJhQfmgw== 2022-09-20T20:06:49.482 INFO:teuthology.orchestra.run.smithi036.stdout:[client.0] 2022-09-20T20:06:49.482 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA6HSpj8a41ChAAGfBxT1Pp/0ylfiC/GHlTMQ== 2022-09-20T20:06:49.483 INFO:teuthology.orchestra.run.smithi036.stdout:[client.2] 2022-09-20T20:06:49.483 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA6HSpjI5OdDRAA0D4JWnMvmXnAXzIEhRvqAA== 2022-09-20T20:06:49.484 INFO:teuthology.orchestra.run.smithi036.stdout:[client.1] 2022-09-20T20:06:49.484 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA6HSpjKsX8EBAA0VMMaiQyh8ppnkaca6xKFQ== 2022-09-20T20:06:49.484 INFO:teuthology.orchestra.run.smithi036.stdout:[client.3] 2022-09-20T20:06:49.485 INFO:teuthology.orchestra.run.smithi036.stdout: key = AQA6HSpjg/uyFhAAVEPW+L9/HB8Ok6oqSrv37g== 2022-09-20T20:06:49.498 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.y] 2022-09-20T20:06:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA5HSpjKyQ/JhAAUr0WiEf3Ir6byeSc/VMU7A== 2022-09-20T20:06:49.499 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.a] 2022-09-20T20:06:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA5HSpjj7unLRAAiVEe8hkQ+5w5pp58yzahzA== 2022-09-20T20:06:49.499 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.c] 2022-09-20T20:06:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA5HSpjDc4YNRAAFPEP0OI1c7F3BBozU153yw== 2022-09-20T20:06:49.500 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.0] 2022-09-20T20:06:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA9HSpjtXsGHxAAHjD2WP7PAVHK7xKkniE+HQ== 2022-09-20T20:06:49.500 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.1] 2022-09-20T20:06:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBAHSpjUj7FHRAAnwKmrJmrZMBcMBxuwiBNTw== 2022-09-20T20:06:49.501 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.2] 2022-09-20T20:06:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBDHSpjVy0vHRAAYiuPX+QHWTGmVXnFGCC78g== 2022-09-20T20:06:49.501 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.3] 2022-09-20T20:06:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBGHSpjOkuLHBAAp/eVepQ7mpC4ArvTA28lnQ== 2022-09-20T20:06:49.502 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.x] 2022-09-20T20:06:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA5HSpjx6E1KhAAwSo6SMtmdCuyay9YaokOGQ== 2022-09-20T20:06:49.502 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.b] 2022-09-20T20:06:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA5HSpj8vUVOhAAkT7gRGCPj8NLuXqjSorJUw== 2022-09-20T20:06:49.503 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.d] 2022-09-20T20:06:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA6HSpjZg7HBRAA3Os/GI42y1xP9Grdis31LA== 2022-09-20T20:06:49.503 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.4] 2022-09-20T20:06:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBMHSpj62pHIBAA6MUdq1cGl4ohg1Xwh6pD9g== 2022-09-20T20:06:49.504 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.5] 2022-09-20T20:06:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBPHSpjh2YyHxAAnEZj9ld68uVOw2u5/3SSIw== 2022-09-20T20:06:49.504 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.6] 2022-09-20T20:06:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBSHSpjq4p9HhAA6IHnvrEAN9PGvHtS2AsbLA== 2022-09-20T20:06:49.504 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.7] 2022-09-20T20:06:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBVHSpjG2wvHRAAjyjEC2nxxLy0EkgJhQfmgw== 2022-09-20T20:06:49.505 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2022-09-20T20:06:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA6HSpj8a41ChAAGfBxT1Pp/0ylfiC/GHlTMQ== 2022-09-20T20:06:49.505 INFO:teuthology.orchestra.run.smithi115.stdout:[client.2] 2022-09-20T20:06:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA6HSpjI5OdDRAA0D4JWnMvmXnAXzIEhRvqAA== 2022-09-20T20:06:49.506 INFO:teuthology.orchestra.run.smithi115.stdout:[client.1] 2022-09-20T20:06:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA6HSpjKsX8EBAA0VMMaiQyh8ppnkaca6xKFQ== 2022-09-20T20:06:49.507 INFO:teuthology.orchestra.run.smithi115.stdout:[client.3] 2022-09-20T20:06:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA6HSpjg/uyFhAAVEPW+L9/HB8Ok6oqSrv37g== 2022-09-20T20:06:49.508 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-09-20T20:06:49.531 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-09-20T20:06:49.602 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-09-20T20:06:49.604 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-09-20T20:06:49.663 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-09-20T20:06:49.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-09-20T20:06:49.722 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:49.725 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:49.778 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:49.781 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:49.834 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:49.836 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:49.889 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:49.891 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:49.944 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-09-20T20:06:49.946 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-09-20T20:06:49.999 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-09-20T20:06:50.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-09-20T20:06:50.057 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-09-20T20:06:50.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-09-20T20:06:50.116 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:50.119 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:50.172 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:50.175 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:50.228 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:50.230 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:50.283 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:50.285 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-20T20:06:50.338 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-20T20:06:50.341 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-20T20:06:50.396 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-20T20:06:50.399 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-20T20:06:50.455 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-20T20:06:50.458 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-20T20:06:50.515 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-20T20:06:50.517 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-20T20:06:50.573 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-09-20T20:06:50.574 DEBUG:teuthology.orchestra.run.smithi036:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-09-20T20:06:50.602 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-09-20T20:06:50.700 DEBUG:teuthology.orchestra.run.smithi036:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-09-20T20:06:50.727 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-09-20T20:06:50.755 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-09-20T20:06:50.852 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-09-20T20:06:50.880 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-09-20T20:06:50.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-09-20T20:06:51.046 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-09-20T20:06:51.073 DEBUG:teuthology.orchestra.run.smithi036:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:51.075 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-09-20T20:06:51.129 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-09-20T20:06:51.130 INFO:tasks.ceph.mon.a:Restarting daemon 2022-09-20T20:06:51.131 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-09-20T20:06:51.134 INFO:tasks.ceph.mon.a:Started 2022-09-20T20:06:51.134 INFO:tasks.ceph.mon.b:Restarting daemon 2022-09-20T20:06:51.134 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-09-20T20:06:51.172 INFO:tasks.ceph.mon.b:Started 2022-09-20T20:06:51.172 INFO:tasks.ceph.mon.c:Restarting daemon 2022-09-20T20:06:51.173 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-09-20T20:06:51.175 INFO:tasks.ceph.mon.c:Started 2022-09-20T20:06:51.175 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-09-20T20:06:51.176 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-09-20T20:06:51.176 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-09-20T20:06:51.179 INFO:tasks.ceph.mgr.y:Started 2022-09-20T20:06:51.179 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-09-20T20:06:51.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-09-20T20:06:51.182 INFO:tasks.ceph.mgr.x:Started 2022-09-20T20:06:51.182 INFO:tasks.ceph:Setting crush tunables to default 2022-09-20T20:06:51.183 DEBUG:teuthology.orchestra.run.smithi036:> sudo ceph --cluster ceph osd crush tunables default 2022-09-20T20:06:51.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.306+0000 7fe579f8b700 1 Processor -- start 2022-09-20T20:06:51.310 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.307+0000 7fe579f8b700 1 -- start start 2022-09-20T20:06:51.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.307+0000 7fe579f8b700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe57406fe20 0x7fe57406cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:51.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.307+0000 7fe579f8b700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe57415f8f0 0x7fe57415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:51.311 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.307+0000 7fe579f8b700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57415fd00 con 0x7fe57406fa50 2022-09-20T20:06:51.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.307+0000 7fe579f8b700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57415f460 con 0x7fe57406fe20 2022-09-20T20:06:51.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.307+0000 7fe579f8b700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57415ee50 con 0x7fe57415f8f0 2022-09-20T20:06:51.312 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.307+0000 7fe572ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe57406fe20 0x7fe57406cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:51.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.307+0000 7fe572ffd700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fe57406fe20 0x7fe57406cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:57826/0 (socket says 172.21.15.36:57826) 2022-09-20T20:06:51.313 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.308+0000 7fe572ffd700 1 -- 172.21.15.36:0/4132964808 learned_addr learned my addr 172.21.15.36:0/4132964808 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:06:51.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.308+0000 7fe573fff700 1 --2- 172.21.15.36:0/4132964808 >> v2:172.21.15.115:3300/0 conn(0x7fe57415f8f0 0x7fe57415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:51.314 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.308+0000 7fe572ffd700 1 -- 172.21.15.36:0/4132964808 >> v2:172.21.15.36:3300/0 conn(0x7fe57406fe20 msgr2=0x7fe57406cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-09-20T20:06:51.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.308+0000 7fe572ffd700 1 -- 172.21.15.36:0/4132964808 >> v2:172.21.15.36:3300/0 conn(0x7fe57406fe20 msgr2=0x7fe57406cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-09-20T20:06:51.315 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.308+0000 7fe572ffd700 1 --2- 172.21.15.36:0/4132964808 >> v2:172.21.15.36:3300/0 conn(0x7fe57406fe20 0x7fe57406cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-09-20T20:06:51.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.308+0000 7fe572ffd700 1 --2- 172.21.15.36:0/4132964808 >> v2:172.21.15.36:3300/0 conn(0x7fe57406fe20 0x7fe57406cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-09-20T20:06:51.316 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.308+0000 7fe573fff700 1 -- 172.21.15.36:0/4132964808 >> v2:172.21.15.115:3300/0 conn(0x7fe57415f8f0 msgr2=0x7fe57415ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-09-20T20:06:51.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.308+0000 7fe573fff700 1 -- 172.21.15.36:0/4132964808 >> v2:172.21.15.115:3300/0 conn(0x7fe57415f8f0 msgr2=0x7fe57415ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-09-20T20:06:51.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.308+0000 7fe573fff700 1 --2- 172.21.15.36:0/4132964808 >> v2:172.21.15.115:3300/0 conn(0x7fe57415f8f0 0x7fe57415ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-09-20T20:06:51.317 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.308+0000 7fe573fff700 1 --2- 172.21.15.36:0/4132964808 >> v2:172.21.15.115:3300/0 conn(0x7fe57415f8f0 0x7fe57415ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-09-20T20:06:51.512 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.507+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 <== mon.1 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942375605 0 0) 0x7fe564001110 con 0x7fe57406fa50 2022-09-20T20:06:51.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.507+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5540035b0 con 0x7fe57406fa50 2022-09-20T20:06:51.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 <== mon.1 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1189205726 0 0) 0x7fe5540035b0 con 0x7fe57406fa50 2022-09-20T20:06:51.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 >> v2:172.21.15.115:3300/0 conn(0x7fe57415f8f0 msgr2=0x7fe57415ace0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-09-20T20:06:51.513 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 --2- 172.21.15.36:0/4132964808 >> v2:172.21.15.115:3300/0 conn(0x7fe57415f8f0 0x7fe57415ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:51.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 >> v2:172.21.15.36:3300/0 conn(0x7fe57406fe20 msgr2=0x7fe57406cd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-09-20T20:06:51.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 --2- 172.21.15.36:0/4132964808 >> v2:172.21.15.36:3300/0 conn(0x7fe57406fe20 0x7fe57406cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:51.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57406d360 con 0x7fe57406fa50 2022-09-20T20:06:51.514 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 <== mon.1 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3393640461 0 0) 0x7fe564001800 con 0x7fe57406fa50 2022-09-20T20:06:51.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 >> v1:172.21.15.115:6789/0 conn(0x7fe57406fa50 legacy=0x7fe5740ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:51.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe554003a00 0x7fe554003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:51.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe554004850 0x7fe554004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:51.515 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe55400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:51.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57415f460 con 0x7fe554005200 2022-09-20T20:06:51.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57415ee50 con 0x7fe554003a00 2022-09-20T20:06:51.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55400c0e0 con 0x7fe554004850 2022-09-20T20:06:51.516 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe573fff700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe55400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:51.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe572ffd700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe554004850 0x7fe554004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:51.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe5737fe700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe554003a00 0x7fe554003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:51.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe573fff700 1 -- 172.21.15.36:0/4132964808 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe554004850 msgr2=0x7fe554004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:51.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe573fff700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe554004850 0x7fe554004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:51.517 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe573fff700 1 -- 172.21.15.36:0/4132964808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe554003a00 msgr2=0x7fe554003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:51.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe573fff700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe554003a00 0x7fe554003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:51.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe573fff700 1 -- 172.21.15.36:0/4132964808 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5540035b0 con 0x7fe554005200 2022-09-20T20:06:51.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.508+0000 7fe573fff700 1 -- 172.21.15.36:0/4132964808 reap_dead start 2022-09-20T20:06:51.518 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.509+0000 7fe573fff700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe55400ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009fd0 tx=0x7fe554005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b292b026aeb20ec server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:51.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.509+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c007e20 con 0x7fe554005200 2022-09-20T20:06:51.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.509+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c003a20 con 0x7fe554005200 2022-09-20T20:06:51.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.509+0000 7fe5727fc700 1 -- 172.21.15.36:0/4132964808 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c003dd0 con 0x7fe554005200 2022-09-20T20:06:51.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.510+0000 7fe579f8b700 1 -- 172.21.15.36:0/4132964808 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 msgr2=0x7fe55400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:51.519 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.510+0000 7fe579f8b700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe55400ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009fd0 tx=0x7fe554005880 comp rx=0 tx=0).stop 2022-09-20T20:06:51.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.510+0000 7fe579f8b700 1 -- 172.21.15.36:0/4132964808 shutdown_connections 2022-09-20T20:06:51.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.510+0000 7fe579f8b700 1 --2- 172.21.15.36:0/4132964808 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe55400ba10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:51.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.510+0000 7fe579f8b700 1 -- 172.21.15.36:0/4132964808 >> 172.21.15.36:0/4132964808 conn(0x7fe5740cde30 msgr2=0x7fe5740ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:51.520 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.510+0000 7fe579f8b700 1 -- 172.21.15.36:0/4132964808 shutdown_connections 2022-09-20T20:06:51.521 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.510+0000 7fe579f8b700 1 -- 172.21.15.36:0/4132964808 wait complete. 2022-09-20T20:06:51.524 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.512+0000 7fe579f8b700 1 Processor -- start 2022-09-20T20:06:51.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.512+0000 7fe579f8b700 1 -- start start 2022-09-20T20:06:51.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe579f8b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe5740c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:51.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe579f8b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5741586f0 0x7fe574158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:51.525 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe579f8b700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe57415dce0 0x7fe57415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:51.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe579f8b700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe574148c80 con 0x7fe554005200 2022-09-20T20:06:51.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe579f8b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740c4720 con 0x7fe5741586f0 2022-09-20T20:06:51.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe579f8b700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57415e690 con 0x7fe57415dce0 2022-09-20T20:06:51.526 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe5737fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe5740c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:51.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe5737fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe5740c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:57856/0 (socket says 172.21.15.36:57856) 2022-09-20T20:06:51.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe5737fe700 1 -- 172.21.15.36:0/2337252095 learned_addr learned my addr 172.21.15.36:0/2337252095 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:06:51.527 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe572ffd700 1 --2- 172.21.15.36:0/2337252095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5741586f0 0x7fe574158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:51.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe573fff700 1 --2- 172.21.15.36:0/2337252095 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe57415dce0 0x7fe57415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:51.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe5737fe700 1 -- 172.21.15.36:0/2337252095 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe57415dce0 msgr2=0x7fe57415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:51.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe5737fe700 1 --2- 172.21.15.36:0/2337252095 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe57415dce0 0x7fe57415e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:51.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe5737fe700 1 -- 172.21.15.36:0/2337252095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5741586f0 msgr2=0x7fe574158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:51.528 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe5737fe700 1 --2- 172.21.15.36:0/2337252095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5741586f0 0x7fe574158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:51.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.513+0000 7fe5737fe700 1 -- 172.21.15.36:0/2337252095 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574147b10 con 0x7fe554005200 2022-09-20T20:06:51.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.514+0000 7fe5737fe700 1 --2- 172.21.15.36:0/2337252095 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe5740c41e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe564000fb0 tx=0x7fe56400fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:51.529 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.515+0000 7fe56bfff700 1 -- 172.21.15.36:0/2337252095 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640052e0 con 0x7fe554005200 2022-09-20T20:06:51.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.515+0000 7fe56bfff700 1 -- 172.21.15.36:0/2337252095 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe564005460 con 0x7fe554005200 2022-09-20T20:06:51.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.515+0000 7fe579f8b700 1 -- 172.21.15.36:0/2337252095 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe574148000 con 0x7fe554005200 2022-09-20T20:06:51.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.516+0000 7fe56bfff700 1 -- 172.21.15.36:0/2337252095 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640038c0 con 0x7fe554005200 2022-09-20T20:06:51.530 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.516+0000 7fe579f8b700 1 -- 172.21.15.36:0/2337252095 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5741485c0 con 0x7fe554005200 2022-09-20T20:06:51.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.516+0000 7fe56bfff700 1 -- 172.21.15.36:0/2337252095 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7fe564003a20 con 0x7fe554005200 2022-09-20T20:06:51.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.516+0000 7fe56bfff700 1 -- 172.21.15.36:0/2337252095 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe56401d8e0 con 0x7fe554005200 2022-09-20T20:06:51.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.517+0000 7fe579f8b700 1 -- 172.21.15.36:0/2337252095 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5740663c0 con 0x7fe554005200 2022-09-20T20:06:51.531 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.519+0000 7fe56bfff700 1 -- 172.21.15.36:0/2337252095 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7fe57415fd00 con 0x7fe554005200 2022-09-20T20:06:51.598 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.597+0000 7fe579f8b700 1 -- 172.21.15.36:0/2337252095 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe57415f0f0 con 0x7fe554005200 2022-09-20T20:06:51.606 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.604+0000 7fe56bfff700 1 -- 172.21.15.36:0/2337252095 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe564003e40 con 0x7fe554005200 2022-09-20T20:06:51.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.605+0000 7fe579f8b700 1 -- 172.21.15.36:0/2337252095 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 msgr2=0x7fe5740c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:51.607 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.605+0000 7fe579f8b700 1 --2- 172.21.15.36:0/2337252095 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe5740c41e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe564000fb0 tx=0x7fe56400fb20 comp rx=0 tx=0).stop 2022-09-20T20:06:51.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.605+0000 7fe579f8b700 1 -- 172.21.15.36:0/2337252095 shutdown_connections 2022-09-20T20:06:51.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.605+0000 7fe579f8b700 1 --2- 172.21.15.36:0/2337252095 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe57415dce0 0x7fe57415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:51.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.605+0000 7fe579f8b700 1 --2- 172.21.15.36:0/2337252095 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe554005200 0x7fe5740c41e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:51.608 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.605+0000 7fe579f8b700 1 --2- 172.21.15.36:0/2337252095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5741586f0 0x7fe574158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:51.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.605+0000 7fe579f8b700 1 -- 172.21.15.36:0/2337252095 >> 172.21.15.36:0/2337252095 conn(0x7fe5740cde30 msgr2=0x7fe5740b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:51.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.605+0000 7fe579f8b700 1 -- 172.21.15.36:0/2337252095 shutdown_connections 2022-09-20T20:06:51.609 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:51.605+0000 7fe579f8b700 1 -- 172.21.15.36:0/2337252095 wait complete. 2022-09-20T20:06:51.609 INFO:teuthology.orchestra.run.smithi036.stderr:adjusted tunables profile to default 2022-09-20T20:06:51.619 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-09-20T20:06:51.619 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:51.620 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-09-20T20:06:51.623 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:51.622+0000 7f7f713b6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-09-20T20:06:51.629 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:51.628+0000 7f77b518b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-09-20T20:06:51.648 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:51.649 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-09-20T20:06:51.685 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:51.684+0000 7f7f713b6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-09-20T20:06:51.697 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:51.696+0000 7f77b518b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-09-20T20:06:51.714 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:51.715 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-09-20T20:06:51.758 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:51.757+0000 7f7f713b6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-09-20T20:06:51.776 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:51.775+0000 7f77b518b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-09-20T20:06:51.782 DEBUG:teuthology.orchestra.run.smithi036:> set -ex 2022-09-20T20:06:51.782 DEBUG:teuthology.orchestra.run.smithi036:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-09-20T20:06:51.835 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:51.833+0000 7f7f713b6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-09-20T20:06:51.850 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:51.851 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2022-09-20T20:06:51.860 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:51.858+0000 7f77b518b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-09-20T20:06:51.876 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:51.877 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2022-09-20T20:06:51.942 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:51.943 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2022-09-20T20:06:51.979 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:51.977+0000 7f7f713b6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-09-20T20:06:52.004 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2022-09-20T20:06:52.005 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2022-09-20T20:06:52.006 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:52.004+0000 7f77b518b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-09-20T20:06:52.071 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 027a07c2-8eec-415a-9e2b-ae85e68a47ad 0 2022-09-20T20:06:52.231 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.229+0000 7f184741c700 1 Processor -- start 2022-09-20T20:06:52.231 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.229+0000 7f184741c700 1 -- start start 2022-09-20T20:06:52.238 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f184741c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f18400ca3d0 0x7f1840070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.239 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f184741c700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f184006a6b0 0x7f18400b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.239 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f184741c700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18400b0570 0x7f184006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.240 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f184741c700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18400b09f0 con 0x7f18400ca3d0 2022-09-20T20:06:52.240 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f184741c700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184006ab50 con 0x7f184006a6b0 2022-09-20T20:06:52.241 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f184741c700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184015b9b0 con 0x7f18400b0570 2022-09-20T20:06:52.241 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18449b7700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f184006a6b0 0x7f18400b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.241 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18459b9700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18400b0570 0x7f184006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.242 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18449b7700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f184006a6b0 0x7f18400b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44578/0 (socket says 172.21.15.115:44578) 2022-09-20T20:06:52.242 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18459b9700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18400b0570 0x7f184006d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3301/0 says I am v2:172.21.15.115:55910/0 (socket says 172.21.15.115:55910) 2022-09-20T20:06:52.242 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18449b7700 1 -- 172.21.15.115:0/2665799892 learned_addr learned my addr 172.21.15.115:0/2665799892 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:52.243 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18451b8700 1 --2- 172.21.15.115:0/2665799892 >> v2:172.21.15.36:3300/0 conn(0x7f18400ca3d0 0x7f1840070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.243 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18459b9700 1 -- 172.21.15.115:0/2665799892 >> v2:172.21.15.115:3300/0 conn(0x7f184006a6b0 msgr2=0x7f18400b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:52.243 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18459b9700 1 --2- 172.21.15.115:0/2665799892 >> v2:172.21.15.115:3300/0 conn(0x7f184006a6b0 0x7f18400b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.244 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18459b9700 1 -- 172.21.15.115:0/2665799892 >> v2:172.21.15.36:3300/0 conn(0x7f18400ca3d0 msgr2=0x7f1840070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:52.244 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18459b9700 1 --2- 172.21.15.115:0/2665799892 >> v2:172.21.15.36:3300/0 conn(0x7f18400ca3d0 0x7f1840070ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.244 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.230+0000 7f18459b9700 1 -- 172.21.15.115:0/2665799892 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184015bc90 con 0x7f18400b0570 2022-09-20T20:06:52.245 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f18449b7700 1 --2- 172.21.15.115:0/2665799892 >> v2:172.21.15.115:3300/0 conn(0x7f184006a6b0 0x7f18400b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-09-20T20:06:52.245 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f18459b9700 1 --2- 172.21.15.115:0/2665799892 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18400b0570 0x7f184006d990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18400ce510 tx=0x7f183800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc8b414702aba63c server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:52.245 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f182ffff700 1 -- 172.21.15.115:0/2665799892 <== mon.2 v2:172.21.15.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183800e070 con 0x7f18400b0570 2022-09-20T20:06:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f182ffff700 1 -- 172.21.15.115:0/2665799892 <== mon.2 v2:172.21.15.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183800bb20 con 0x7f18400b0570 2022-09-20T20:06:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f182ffff700 1 -- 172.21.15.115:0/2665799892 <== mon.2 v2:172.21.15.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838012430 con 0x7f18400b0570 2022-09-20T20:06:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f184741c700 1 -- 172.21.15.115:0/2665799892 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18400b0570 msgr2=0x7f184006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.247 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f184741c700 1 --2- 172.21.15.115:0/2665799892 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18400b0570 0x7f184006d990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18400ce510 tx=0x7f183800b330 comp rx=0 tx=0).stop 2022-09-20T20:06:52.247 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f184741c700 1 -- 172.21.15.115:0/2665799892 shutdown_connections 2022-09-20T20:06:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f184741c700 1 --2- 172.21.15.115:0/2665799892 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18400b0570 0x7f184006d990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f184741c700 1 --2- 172.21.15.115:0/2665799892 >> v2:172.21.15.36:3300/0 conn(0x7f18400ca3d0 0x7f1840070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f184741c700 1 --2- 172.21.15.115:0/2665799892 >> v2:172.21.15.115:3300/0 conn(0x7f184006a6b0 0x7f18400b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.231+0000 7f184741c700 1 -- 172.21.15.115:0/2665799892 >> 172.21.15.115:0/2665799892 conn(0x7f18400cb030 msgr2=0x7f18400cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.232+0000 7f184741c700 1 -- 172.21.15.115:0/2665799892 shutdown_connections 2022-09-20T20:06:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.232+0000 7f184741c700 1 -- 172.21.15.115:0/2665799892 wait complete. 2022-09-20T20:06:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.233+0000 7f184741c700 1 Processor -- start 2022-09-20T20:06:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.233+0000 7f184741c700 1 -- start start 2022-09-20T20:06:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.233+0000 7f184741c700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f184006a6b0 0x7f18400aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.233+0000 7f184741c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18400b0570 0x7f18401f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.233+0000 7f184741c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18400ca3d0 0x7f18401f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.233+0000 7f184741c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18400b2d00 con 0x7f18400ca3d0 2022-09-20T20:06:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.233+0000 7f184741c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1840157470 con 0x7f18400b0570 2022-09-20T20:06:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.233+0000 7f184741c700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18400b2fc0 con 0x7f184006a6b0 2022-09-20T20:06:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18451b8700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f184006a6b0 0x7f18400aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18449b7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18400b0570 0x7f18401f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18451b8700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f184006a6b0 0x7f18400aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3301/0 says I am v2:172.21.15.115:55912/0 (socket says 172.21.15.115:55912) 2022-09-20T20:06:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18451b8700 1 -- 172.21.15.115:0/1698518129 learned_addr learned my addr 172.21.15.115:0/1698518129 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18459b9700 1 --2- 172.21.15.115:0/1698518129 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18400ca3d0 0x7f18401f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18451b8700 1 -- 172.21.15.115:0/1698518129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18400b0570 msgr2=0x7f18401f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18451b8700 1 --2- 172.21.15.115:0/1698518129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18400b0570 0x7f18401f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18451b8700 1 -- 172.21.15.115:0/1698518129 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18400ca3d0 msgr2=0x7f18401f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18451b8700 1 --2- 172.21.15.115:0/1698518129 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18400ca3d0 0x7f18401f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18451b8700 1 -- 172.21.15.115:0/1698518129 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830009550 con 0x7f184006a6b0 2022-09-20T20:06:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18459b9700 1 --2- 172.21.15.115:0/1698518129 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18400ca3d0 0x7f18401f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-09-20T20:06:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f18451b8700 1 --2- 172.21.15.115:0/1698518129 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f184006a6b0 0x7f18400aaa10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f183c00c7f0 tx=0x7f183c00cbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f182dffb700 1 -- 172.21.15.115:0/1698518129 <== mon.2 v2:172.21.15.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c011800 con 0x7f184006a6b0 2022-09-20T20:06:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f182dffb700 1 -- 172.21.15.115:0/1698518129 <== mon.2 v2:172.21.15.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c011960 con 0x7f184006a6b0 2022-09-20T20:06:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f184741c700 1 -- 172.21.15.115:0/1698518129 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1838009a70 con 0x7f184006a6b0 2022-09-20T20:06:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f182dffb700 1 -- 172.21.15.115:0/1698518129 <== mon.2 v2:172.21.15.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c011cf0 con 0x7f184006a6b0 2022-09-20T20:06:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f182dffb700 1 -- 172.21.15.115:0/1698518129 <== mon.2 v2:172.21.15.115:3301/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7f183c017030 con 0x7f184006a6b0 2022-09-20T20:06:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f184741c700 1 -- 172.21.15.115:0/1698518129 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1840156bd0 con 0x7f184006a6b0 2022-09-20T20:06:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.234+0000 7f182dffb700 1 -- 172.21.15.115:0/1698518129 <== mon.2 v2:172.21.15.115:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f183c0233a0 con 0x7f184006a6b0 2022-09-20T20:06:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.235+0000 7f184741c700 1 -- 172.21.15.115:0/1698518129 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18400663c0 con 0x7f184006a6b0 2022-09-20T20:06:52.260 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.237+0000 7f182dffb700 1 -- 172.21.15.115:0/1698518129 <== mon.2 v2:172.21.15.115:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7f183c01f060 con 0x7f184006a6b0 2022-09-20T20:06:52.315 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:52.313+0000 7f7f713b6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-09-20T20:06:52.321 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.319+0000 7f184741c700 1 -- 172.21.15.115:0/1698518129 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_command({"prefix": "osd new", "uuid": "027a07c2-8eec-415a-9e2b-ae85e68a47ad", "id": 0} v 0) v1 -- 0x7f184015bf60 con 0x7f184006a6b0 2022-09-20T20:06:52.326 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.325+0000 7f182dffb700 1 -- 172.21.15.115:0/1698518129 <== mon.2 v2:172.21.15.115:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "027a07c2-8eec-415a-9e2b-ae85e68a47ad", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f183c015070 con 0x7f184006a6b0 2022-09-20T20:06:52.327 INFO:teuthology.orchestra.run.smithi115.stdout:0 2022-09-20T20:06:52.329 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.326+0000 7f184741c700 1 -- 172.21.15.115:0/1698518129 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f184006a6b0 msgr2=0x7f18400aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.329 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.326+0000 7f184741c700 1 --2- 172.21.15.115:0/1698518129 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f184006a6b0 0x7f18400aaa10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f183c00c7f0 tx=0x7f183c00cbb0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.329 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.326+0000 7f184741c700 1 -- 172.21.15.115:0/1698518129 shutdown_connections 2022-09-20T20:06:52.329 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.326+0000 7f184741c700 1 --2- 172.21.15.115:0/1698518129 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18400ca3d0 0x7f18401f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.326+0000 7f184741c700 1 --2- 172.21.15.115:0/1698518129 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f184006a6b0 0x7f18400aaa10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.326+0000 7f184741c700 1 --2- 172.21.15.115:0/1698518129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18400b0570 0x7f18401f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.326+0000 7f184741c700 1 -- 172.21.15.115:0/1698518129 >> 172.21.15.115:0/1698518129 conn(0x7f18400cb030 msgr2=0x7f184015cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:52.330 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.326+0000 7f184741c700 1 -- 172.21.15.115:0/1698518129 shutdown_connections 2022-09-20T20:06:52.331 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.326+0000 7f184741c700 1 -- 172.21.15.115:0/1698518129 wait complete. 2022-09-20T20:06:52.345 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 73600a5c-a12c-460c-80ec-6fcd76cc6f3f 1 2022-09-20T20:06:52.395 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:52.393+0000 7f7f713b6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-09-20T20:06:52.468 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.466+0000 7ff9e544d700 1 Processor -- start 2022-09-20T20:06:52.469 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.467+0000 7ff9e544d700 1 -- start start 2022-09-20T20:06:52.472 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.467+0000 7ff9e544d700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff9e00cf0d0 0x7ff9e00c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.472 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.467+0000 7ff9e544d700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e00b2640 con 0x7ff9e00cc1a0 2022-09-20T20:06:52.472 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.467+0000 7ff9e544d700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e00d0080 con 0x7ff9e00cf4a0 2022-09-20T20:06:52.473 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.467+0000 7ff9e544d700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e00cbec0 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.473 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.467+0000 7ff9deffd700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff9e00cf0d0 0x7ff9e00c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.473 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.467+0000 7ff9deffd700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff9e00cf0d0 0x7ff9e00c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44588/0 (socket says 172.21.15.115:44588) 2022-09-20T20:06:52.473 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.467+0000 7ff9de7fc700 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff9e00cf4a0 0x7ff9e00cb620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48570/0 (socket says 172.21.15.115:48570) 2022-09-20T20:06:52.474 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.467+0000 7ff9deffd700 1 -- 172.21.15.115:0/2953209188 learned_addr learned my addr 172.21.15.115:0/2953209188 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:52.474 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.468+0000 7ff9deffd700 1 -- 172.21.15.115:0/2953209188 >> v1:172.21.15.115:6789/0 conn(0x7ff9e00cf4a0 legacy=0x7ff9e00cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.474 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.468+0000 7ff9deffd700 1 -- 172.21.15.115:0/2953209188 >> v1:172.21.15.36:6789/0 conn(0x7ff9e00cc1a0 legacy=0x7ff9e00cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.474 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.468+0000 7ff9deffd700 1 -- 172.21.15.115:0/2953209188 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00cfa30 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.475 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.468+0000 7ff9deffd700 1 --2- 172.21.15.115:0/2953209188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e00cf0d0 0x7ff9e00c9e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00098f0 tx=0x7ff9d0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c40ae4d6be434315 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:52.475 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.468+0000 7ff9ddffb700 1 -- 172.21.15.115:0/2953209188 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000b8a0 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.475 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.468+0000 7ff9ddffb700 1 -- 172.21.15.115:0/2953209188 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d000ba00 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.475 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.468+0000 7ff9ddffb700 1 -- 172.21.15.115:0/2953209188 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000bd90 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.476 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.469+0000 7ff9e544d700 1 -- 172.21.15.115:0/2953209188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e00cf0d0 msgr2=0x7ff9e00c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.476 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.469+0000 7ff9e544d700 1 --2- 172.21.15.115:0/2953209188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e00cf0d0 0x7ff9e00c9e40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00098f0 tx=0x7ff9d0009c00 comp rx=0 tx=0).stop 2022-09-20T20:06:52.476 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.469+0000 7ff9e544d700 1 -- 172.21.15.115:0/2953209188 shutdown_connections 2022-09-20T20:06:52.476 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.469+0000 7ff9e544d700 1 --2- 172.21.15.115:0/2953209188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e00cf0d0 0x7ff9e00c9e40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.477 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.469+0000 7ff9e544d700 1 -- 172.21.15.115:0/2953209188 >> 172.21.15.115:0/2953209188 conn(0x7ff9e006d190 msgr2=0x7ff9e006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:52.477 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.469+0000 7ff9e544d700 1 -- 172.21.15.115:0/2953209188 shutdown_connections 2022-09-20T20:06:52.477 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.469+0000 7ff9e544d700 1 -- 172.21.15.115:0/2953209188 wait complete. 2022-09-20T20:06:52.477 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.470+0000 7ff9e544d700 1 Processor -- start 2022-09-20T20:06:52.478 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.470+0000 7ff9e544d700 1 -- start start 2022-09-20T20:06:52.481 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9e544d700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7ff9e00cc1a0 0x7ff9e015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.481 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9e544d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e00cf0d0 0x7ff9e0155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9e544d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9e00cf4a0 0x7ff9e0157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9e544d700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0155930 con 0x7ff9e00cf4a0 2022-09-20T20:06:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9e544d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0151460 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9e544d700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0151740 con 0x7ff9e00cc1a0 2022-09-20T20:06:52.483 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9deffd700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7ff9e00cc1a0 0x7ff9e015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.483 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9deffd700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7ff9e00cc1a0 0x7ff9e015bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3301/0 says I am v2:172.21.15.115:55924/0 (socket says 172.21.15.115:55924) 2022-09-20T20:06:52.483 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9deffd700 1 -- 172.21.15.115:0/3275902585 learned_addr learned my addr 172.21.15.115:0/3275902585 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:52.483 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9de7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e00cf0d0 0x7ff9e0155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.483 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9df7fe700 1 --2- 172.21.15.115:0/3275902585 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9e00cf4a0 0x7ff9e0157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.484 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9de7fc700 1 -- 172.21.15.115:0/3275902585 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7ff9e00cc1a0 msgr2=0x7ff9e015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.484 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9de7fc700 1 --2- 172.21.15.115:0/3275902585 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7ff9e00cc1a0 0x7ff9e015bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.484 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9de7fc700 1 -- 172.21.15.115:0/3275902585 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9e00cf4a0 msgr2=0x7ff9e0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9de7fc700 1 --2- 172.21.15.115:0/3275902585 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9e00cf4a0 0x7ff9e0157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.485 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9de7fc700 1 -- 172.21.15.115:0/3275902585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d0009550 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.486 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9de7fc700 1 --2- 172.21.15.115:0/3275902585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e00cf0d0 0x7ff9e0155040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c3c0 tx=0x7ff9d400c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:52.486 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.471+0000 7ff9c77fe700 1 -- 172.21.15.115:0/3275902585 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4004a70 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.487 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.472+0000 7ff9c77fe700 1 -- 172.21.15.115:0/3275902585 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d4004bd0 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.487 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.472+0000 7ff9e544d700 1 -- 172.21.15.115:0/3275902585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e0155b80 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.488 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.472+0000 7ff9c77fe700 1 -- 172.21.15.115:0/3275902585 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40070d0 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.488 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.472+0000 7ff9e544d700 1 -- 172.21.15.115:0/3275902585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e0155580 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.489 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.472+0000 7ff9c77fe700 1 -- 172.21.15.115:0/3275902585 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7ff9d4010070 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.489 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.472+0000 7ff9c77fe700 1 -- 172.21.15.115:0/3275902585 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7ff9d4013b10 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.490 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.472+0000 7ff9e544d700 1 -- 172.21.15.115:0/3275902585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e01560a0 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.490 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.475+0000 7ff9c77fe700 1 -- 172.21.15.115:0/3275902585 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7ff9e01560a0 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.491 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:52.479+0000 7f77b518b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-09-20T20:06:52.532 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:52.531+0000 7f7f713b6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-09-20T20:06:52.558 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:52.556+0000 7f77b518b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-09-20T20:06:52.560 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.558+0000 7ff9e544d700 1 -- 172.21.15.115:0/3275902585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "73600a5c-a12c-460c-80ec-6fcd76cc6f3f", "id": 1} v 0) v1 -- 0x7ff9e00663c0 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.567 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.565+0000 7ff9c77fe700 1 -- 172.21.15.115:0/3275902585 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "73600a5c-a12c-460c-80ec-6fcd76cc6f3f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff9d4007680 con 0x7ff9e00cf0d0 2022-09-20T20:06:52.568 INFO:teuthology.orchestra.run.smithi115.stdout:1 2022-09-20T20:06:52.571 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.566+0000 7ff9e544d700 1 -- 172.21.15.115:0/3275902585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e00cf0d0 msgr2=0x7ff9e0155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.571 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.566+0000 7ff9e544d700 1 --2- 172.21.15.115:0/3275902585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e00cf0d0 0x7ff9e0155040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c3c0 tx=0x7ff9d400c6d0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.572 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.566+0000 7ff9e544d700 1 -- 172.21.15.115:0/3275902585 shutdown_connections 2022-09-20T20:06:52.572 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.566+0000 7ff9e544d700 1 --2- 172.21.15.115:0/3275902585 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7ff9e00cf4a0 0x7ff9e0157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.572 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.566+0000 7ff9e544d700 1 --2- 172.21.15.115:0/3275902585 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7ff9e00cc1a0 0x7ff9e015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.573 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.566+0000 7ff9e544d700 1 --2- 172.21.15.115:0/3275902585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9e00cf0d0 0x7ff9e0155040 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.573 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.566+0000 7ff9e544d700 1 -- 172.21.15.115:0/3275902585 >> 172.21.15.115:0/3275902585 conn(0x7ff9e006d190 msgr2=0x7ff9e015ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:52.574 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.566+0000 7ff9e544d700 1 -- 172.21.15.115:0/3275902585 shutdown_connections 2022-09-20T20:06:52.574 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.566+0000 7ff9e544d700 1 -- 172.21.15.115:0/3275902585 wait complete. 2022-09-20T20:06:52.580 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 76ffb9bc-7fa7-4cdd-8190-8caebf90ccc4 2 2022-09-20T20:06:52.618 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:52.616+0000 7f7f713b6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-09-20T20:06:52.689 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:52.687+0000 7f77b518b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-09-20T20:06:52.703 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.701+0000 7fea7491e700 1 Processor -- start 2022-09-20T20:06:52.703 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.701+0000 7fea7491e700 1 -- start start 2022-09-20T20:06:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea7491e700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea700c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea7491e700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fea700cf4a0 0x7fea700cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea7491e700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fea700cc1a0 0x7fea700cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea7491e700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea700b2640 con 0x7fea700cc1a0 2022-09-20T20:06:52.708 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea7491e700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea700d0080 con 0x7fea700cf4a0 2022-09-20T20:06:52.708 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea7491e700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea700ccb60 con 0x7fea700cf0d0 2022-09-20T20:06:52.708 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea6dd9b700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fea700cf4a0 0x7fea700cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.708 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea6e59c700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea700c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.708 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea6dd9b700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fea700cf4a0 0x7fea700cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44600/0 (socket says 172.21.15.115:44600) 2022-09-20T20:06:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea6dd9b700 1 -- 172.21.15.115:0/820939296 learned_addr learned my addr 172.21.15.115:0/820939296 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea6ed9d700 1 --2- 172.21.15.115:0/820939296 >> v2:172.21.15.36:3300/0 conn(0x7fea700cc1a0 0x7fea700cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea6e59c700 1 -- 172.21.15.115:0/820939296 >> v2:172.21.15.115:3300/0 conn(0x7fea700cf4a0 msgr2=0x7fea700cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea6e59c700 1 --2- 172.21.15.115:0/820939296 >> v2:172.21.15.115:3300/0 conn(0x7fea700cf4a0 0x7fea700cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea6e59c700 1 -- 172.21.15.115:0/820939296 >> v2:172.21.15.36:3300/0 conn(0x7fea700cc1a0 msgr2=0x7fea700cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea6e59c700 1 --2- 172.21.15.115:0/820939296 >> v2:172.21.15.36:3300/0 conn(0x7fea700cc1a0 0x7fea700cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.702+0000 7fea6e59c700 1 -- 172.21.15.115:0/820939296 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea700cbec0 con 0x7fea700cf0d0 2022-09-20T20:06:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea6e59c700 1 --2- 172.21.15.115:0/820939296 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea700c9e40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea700d0800 tx=0x7fea60009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb86add87fe7cdcf server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea6d59a700 1 -- 172.21.15.115:0/820939296 <== mon.2 v2:172.21.15.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60011830 con 0x7fea700cf0d0 2022-09-20T20:06:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea6d59a700 1 -- 172.21.15.115:0/820939296 <== mon.2 v2:172.21.15.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea60011990 con 0x7fea700cf0d0 2022-09-20T20:06:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea6d59a700 1 -- 172.21.15.115:0/820939296 <== mon.2 v2:172.21.15.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60011d20 con 0x7fea700cf0d0 2022-09-20T20:06:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea7491e700 1 -- 172.21.15.115:0/820939296 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 msgr2=0x7fea700c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea7491e700 1 --2- 172.21.15.115:0/820939296 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea700c9e40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea700d0800 tx=0x7fea60009b20 comp rx=0 tx=0).stop 2022-09-20T20:06:52.712 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea7491e700 1 -- 172.21.15.115:0/820939296 shutdown_connections 2022-09-20T20:06:52.712 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea7491e700 1 --2- 172.21.15.115:0/820939296 >> v2:172.21.15.36:3300/0 conn(0x7fea700cc1a0 0x7fea700cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.712 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea7491e700 1 --2- 172.21.15.115:0/820939296 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea700c9e40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.712 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea7491e700 1 --2- 172.21.15.115:0/820939296 >> v2:172.21.15.115:3300/0 conn(0x7fea700cf4a0 0x7fea700cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.703+0000 7fea7491e700 1 -- 172.21.15.115:0/820939296 >> 172.21.15.115:0/820939296 conn(0x7fea7006d190 msgr2=0x7fea7006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.704+0000 7fea7491e700 1 -- 172.21.15.115:0/820939296 shutdown_connections 2022-09-20T20:06:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.704+0000 7fea7491e700 1 -- 172.21.15.115:0/820939296 wait complete. 2022-09-20T20:06:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.705+0000 7fea7491e700 1 Processor -- start 2022-09-20T20:06:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.705+0000 7fea7491e700 1 -- start start 2022-09-20T20:06:52.716 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea7491e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fea700cc1a0 0x7fea700c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea7491e700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea70157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea7491e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea700cf4a0 0x7fea700b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea7491e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700c7180 con 0x7fea700cc1a0 2022-09-20T20:06:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea7491e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700c7460 con 0x7fea700cf4a0 2022-09-20T20:06:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea7491e700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea700c7740 con 0x7fea700cf0d0 2022-09-20T20:06:52.718 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6dd9b700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea70157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.718 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6dd9b700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea70157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3301/0 says I am v2:172.21.15.115:55938/0 (socket says 172.21.15.115:55938) 2022-09-20T20:06:52.718 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6dd9b700 1 -- 172.21.15.115:0/1981190932 learned_addr learned my addr 172.21.15.115:0/1981190932 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:52.718 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea700cf4a0 0x7fea700b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6e59c700 1 --2- 172.21.15.115:0/1981190932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fea700cc1a0 0x7fea700c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6dd9b700 1 -- 172.21.15.115:0/1981190932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea700cf4a0 msgr2=0x7fea700b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6dd9b700 1 --2- 172.21.15.115:0/1981190932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea700cf4a0 0x7fea700b01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6dd9b700 1 -- 172.21.15.115:0/1981190932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fea700cc1a0 msgr2=0x7fea700c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6dd9b700 1 --2- 172.21.15.115:0/1981190932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fea700cc1a0 0x7fea700c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.720 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6dd9b700 1 -- 172.21.15.115:0/1981190932 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea600094a0 con 0x7fea700cf0d0 2022-09-20T20:06:52.720 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea6dd9b700 1 --2- 172.21.15.115:0/1981190932 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea70157020 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea6400e910 tx=0x7fea6400ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:52.720 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea5effd700 1 -- 172.21.15.115:0/1981190932 <== mon.2 v2:172.21.15.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea640096a0 con 0x7fea700cf0d0 2022-09-20T20:06:52.720 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea5effd700 1 -- 172.21.15.115:0/1981190932 <== mon.2 v2:172.21.15.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea64013680 con 0x7fea700cf0d0 2022-09-20T20:06:52.721 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea7491e700 1 -- 172.21.15.115:0/1981190932 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea700c7a20 con 0x7fea700cf0d0 2022-09-20T20:06:52.721 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea5effd700 1 -- 172.21.15.115:0/1981190932 <== mon.2 v2:172.21.15.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64013a30 con 0x7fea700cf0d0 2022-09-20T20:06:52.721 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.706+0000 7fea7491e700 1 -- 172.21.15.115:0/1981190932 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea700c6e90 con 0x7fea700cf0d0 2022-09-20T20:06:52.721 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.707+0000 7fea5effd700 1 -- 172.21.15.115:0/1981190932 <== mon.2 v2:172.21.15.115:3301/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7fea64019030 con 0x7fea700cf0d0 2022-09-20T20:06:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.707+0000 7fea5effd700 1 -- 172.21.15.115:0/1981190932 <== mon.2 v2:172.21.15.115:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fea64005de0 con 0x7fea700cf0d0 2022-09-20T20:06:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.707+0000 7fea7491e700 1 -- 172.21.15.115:0/1981190932 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea700c6660 con 0x7fea700cf0d0 2022-09-20T20:06:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.709+0000 7fea5effd700 1 -- 172.21.15.115:0/1981190932 <== mon.2 v2:172.21.15.115:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7fea64021060 con 0x7fea700cf0d0 2022-09-20T20:06:52.772 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:52.770+0000 7f77b518b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-09-20T20:06:52.794 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.792+0000 7fea7491e700 1 -- 172.21.15.115:0/1981190932 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_command({"prefix": "osd new", "uuid": "76ffb9bc-7fa7-4cdd-8190-8caebf90ccc4", "id": 2} v 0) v1 -- 0x7fea700663c0 con 0x7fea700cf0d0 2022-09-20T20:06:52.801 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.800+0000 7fea5effd700 1 -- 172.21.15.115:0/1981190932 <== mon.2 v2:172.21.15.115:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "76ffb9bc-7fa7-4cdd-8190-8caebf90ccc4", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fea64017070 con 0x7fea700cf0d0 2022-09-20T20:06:52.802 INFO:teuthology.orchestra.run.smithi115.stdout:2 2022-09-20T20:06:52.803 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.801+0000 7fea7491e700 1 -- 172.21.15.115:0/1981190932 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 msgr2=0x7fea70157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.803 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.801+0000 7fea7491e700 1 --2- 172.21.15.115:0/1981190932 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea70157020 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea6400e910 tx=0x7fea6400ec20 comp rx=0 tx=0).stop 2022-09-20T20:06:52.803 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.801+0000 7fea7491e700 1 -- 172.21.15.115:0/1981190932 shutdown_connections 2022-09-20T20:06:52.804 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.801+0000 7fea7491e700 1 --2- 172.21.15.115:0/1981190932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea700cf4a0 0x7fea700b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.804 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.801+0000 7fea7491e700 1 --2- 172.21.15.115:0/1981190932 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fea700cc1a0 0x7fea700c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.804 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.801+0000 7fea7491e700 1 --2- 172.21.15.115:0/1981190932 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fea700cf0d0 0x7fea70157020 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.804 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.801+0000 7fea7491e700 1 -- 172.21.15.115:0/1981190932 >> 172.21.15.115:0/1981190932 conn(0x7fea7006d190 msgr2=0x7fea70158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:52.805 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.801+0000 7fea7491e700 1 -- 172.21.15.115:0/1981190932 shutdown_connections 2022-09-20T20:06:52.805 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.801+0000 7fea7491e700 1 -- 172.21.15.115:0/1981190932 wait complete. 2022-09-20T20:06:52.815 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new dd9de39a-4958-4162-9a85-4d950e22837c 3 2022-09-20T20:06:52.974 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.973+0000 7f8ac90b2700 1 Processor -- start 2022-09-20T20:06:52.975 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.973+0000 7f8ac90b2700 1 -- start start 2022-09-20T20:06:52.978 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.973+0000 7f8ac90b2700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.978 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.973+0000 7f8ac90b2700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8ac40ad5a0 0x7f8ac40cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.978 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.973+0000 7f8ac90b2700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac415c9a0 con 0x7f8ac415c450 2022-09-20T20:06:52.979 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.973+0000 7f8ac90b2700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac40c9750 con 0x7f8ac40ad5a0 2022-09-20T20:06:52.979 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.973+0000 7f8ac90b2700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40cabf0 con 0x7f8ac415b500 2022-09-20T20:06:52.979 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac359e700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8ac40ad5a0 0x7f8ac40cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.979 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac2d9d700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.980 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac359e700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8ac40ad5a0 0x7f8ac40cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44616/0 (socket says 172.21.15.115:44616) 2022-09-20T20:06:52.980 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac359e700 1 -- 172.21.15.115:0/2902191304 learned_addr learned my addr 172.21.15.115:0/2902191304 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:52.980 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac2d9d700 1 -- 172.21.15.115:0/2902191304 >> v2:172.21.15.115:3300/0 conn(0x7f8ac40ad5a0 msgr2=0x7f8ac40cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:52.980 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac2d9d700 1 --2- 172.21.15.115:0/2902191304 >> v2:172.21.15.115:3300/0 conn(0x7f8ac40ad5a0 0x7f8ac40cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.980 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac2d9d700 1 -- 172.21.15.115:0/2902191304 >> v1:172.21.15.115:6789/0 conn(0x7f8ac415c450 legacy=0x7f8ac40cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.981 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac2d9d700 1 -- 172.21.15.115:0/2902191304 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac40c8e20 con 0x7f8ac415b500 2022-09-20T20:06:52.981 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac1d9b700 1 -- 172.21.15.115:0/2902191304 <== mon.1 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331873587 0 0) 0x7f8ac415c9a0 con 0x7f8ac415c450 2022-09-20T20:06:52.981 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac359e700 1 --2- 172.21.15.115:0/2902191304 >> v2:172.21.15.115:3300/0 conn(0x7f8ac40ad5a0 0x7f8ac40cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-09-20T20:06:52.981 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac2d9d700 1 --2- 172.21.15.115:0/2902191304 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac40cc200 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009840 tx=0x7f8ab4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1685770ed0318a server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:52.981 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac1d9b700 1 -- 172.21.15.115:0/2902191304 <== mon.2 v2:172.21.15.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400d980 con 0x7f8ac415b500 2022-09-20T20:06:52.982 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac1d9b700 1 -- 172.21.15.115:0/2902191304 <== mon.2 v2:172.21.15.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab400dae0 con 0x7f8ac415b500 2022-09-20T20:06:52.982 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.974+0000 7f8ac1d9b700 1 -- 172.21.15.115:0/2902191304 <== mon.2 v2:172.21.15.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400de70 con 0x7f8ac415b500 2022-09-20T20:06:52.982 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.975+0000 7f8ac90b2700 1 -- 172.21.15.115:0/2902191304 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 msgr2=0x7f8ac40cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.982 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.975+0000 7f8ac90b2700 1 --2- 172.21.15.115:0/2902191304 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac40cc200 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009840 tx=0x7f8ab4009b50 comp rx=0 tx=0).stop 2022-09-20T20:06:52.983 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.975+0000 7f8ac90b2700 1 -- 172.21.15.115:0/2902191304 shutdown_connections 2022-09-20T20:06:52.983 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.975+0000 7f8ac90b2700 1 --2- 172.21.15.115:0/2902191304 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac40cc200 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.983 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.975+0000 7f8ac90b2700 1 --2- 172.21.15.115:0/2902191304 >> v2:172.21.15.115:3300/0 conn(0x7f8ac40ad5a0 0x7f8ac40cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.983 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.975+0000 7f8ac90b2700 1 -- 172.21.15.115:0/2902191304 >> 172.21.15.115:0/2902191304 conn(0x7f8ac406a6d0 msgr2=0x7f8ac40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:52.984 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.975+0000 7f8ac90b2700 1 -- 172.21.15.115:0/2902191304 shutdown_connections 2022-09-20T20:06:52.984 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.975+0000 7f8ac90b2700 1 -- 172.21.15.115:0/2902191304 wait complete. 2022-09-20T20:06:52.984 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.976+0000 7f8ac90b2700 1 Processor -- start 2022-09-20T20:06:52.984 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.976+0000 7f8ac90b2700 1 -- start start 2022-09-20T20:06:52.987 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac90b2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ac40ad5a0 0x7f8ac4151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.988 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac90b2700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac41522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.988 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac90b2700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8ac415c450 0x7f8ac40ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:52.988 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac90b2700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40ad320 con 0x7f8ac415c450 2022-09-20T20:06:52.988 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac90b2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac41520b0 con 0x7f8ac40ad5a0 2022-09-20T20:06:52.989 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac90b2700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40ac7a0 con 0x7f8ac415b500 2022-09-20T20:06:52.989 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac259c700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac41522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.989 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac259c700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac41522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3301/0 says I am v2:172.21.15.115:55950/0 (socket says 172.21.15.115:55950) 2022-09-20T20:06:52.989 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac259c700 1 -- 172.21.15.115:0/661687678 learned_addr learned my addr 172.21.15.115:0/661687678 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:52.990 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac2d9d700 1 --2- 172.21.15.115:0/661687678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ac40ad5a0 0x7f8ac4151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.990 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac359e700 1 --2- 172.21.15.115:0/661687678 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8ac415c450 0x7f8ac40ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:52.991 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac259c700 1 -- 172.21.15.115:0/661687678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ac40ad5a0 msgr2=0x7f8ac4151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.991 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac259c700 1 --2- 172.21.15.115:0/661687678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ac40ad5a0 0x7f8ac4151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.992 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac259c700 1 -- 172.21.15.115:0/661687678 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8ac415c450 msgr2=0x7f8ac40ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:52.992 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac259c700 1 --2- 172.21.15.115:0/661687678 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8ac415c450 0x7f8ac40ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:52.992 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.977+0000 7f8ac259c700 1 -- 172.21.15.115:0/661687678 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc009b30 con 0x7f8ac415b500 2022-09-20T20:06:52.993 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.978+0000 7f8ac259c700 1 --2- 172.21.15.115:0/661687678 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac41522b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c550 tx=0x7f8ab800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:52.993 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.978+0000 7f8aab7fe700 1 -- 172.21.15.115:0/661687678 <== mon.2 v2:172.21.15.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80054e0 con 0x7f8ac415b500 2022-09-20T20:06:52.993 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.978+0000 7f8aab7fe700 1 -- 172.21.15.115:0/661687678 <== mon.2 v2:172.21.15.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab8003680 con 0x7f8ac415b500 2022-09-20T20:06:52.994 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.978+0000 7f8ac90b2700 1 -- 172.21.15.115:0/661687678 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab40094a0 con 0x7f8ac415b500 2022-09-20T20:06:52.994 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.978+0000 7f8aab7fe700 1 -- 172.21.15.115:0/661687678 <== mon.2 v2:172.21.15.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8003a30 con 0x7f8ac415b500 2022-09-20T20:06:52.994 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.978+0000 7f8ac90b2700 1 -- 172.21.15.115:0/661687678 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac40acd60 con 0x7f8ac415b500 2022-09-20T20:06:52.995 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.978+0000 7f8aab7fe700 1 -- 172.21.15.115:0/661687678 <== mon.2 v2:172.21.15.115:3301/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7f8ab8019030 con 0x7f8ac415b500 2022-09-20T20:06:52.995 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.978+0000 7f8aab7fe700 1 -- 172.21.15.115:0/661687678 <== mon.2 v2:172.21.15.115:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f8ab8012890 con 0x7f8ac415b500 2022-09-20T20:06:52.996 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.979+0000 7f8ac90b2700 1 -- 172.21.15.115:0/661687678 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac40718e0 con 0x7f8ac415b500 2022-09-20T20:06:52.996 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:52.981+0000 7f8aab7fe700 1 -- 172.21.15.115:0/661687678 <== mon.2 v2:172.21.15.115:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7f8ab8017070 con 0x7f8ac415b500 2022-09-20T20:06:53.062 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:53.060+0000 7f7f713b6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-09-20T20:06:53.065 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.063+0000 7f8ac90b2700 1 -- 172.21.15.115:0/661687678 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dd9de39a-4958-4162-9a85-4d950e22837c", "id": 3} v 0) v1 -- 0x7f8ac40c9100 con 0x7f8ac415b500 2022-09-20T20:06:53.069 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.067+0000 7f8aab7fe700 1 -- 172.21.15.115:0/661687678 <== mon.2 v2:172.21.15.115:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd9de39a-4958-4162-9a85-4d950e22837c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8ab8012400 con 0x7f8ac415b500 2022-09-20T20:06:53.069 INFO:teuthology.orchestra.run.smithi115.stdout:3 2022-09-20T20:06:53.070 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.068+0000 7f8ac90b2700 1 -- 172.21.15.115:0/661687678 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 msgr2=0x7f8ac41522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.071 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.068+0000 7f8ac90b2700 1 --2- 172.21.15.115:0/661687678 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac41522b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c550 tx=0x7f8ab800c860 comp rx=0 tx=0).stop 2022-09-20T20:06:53.071 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.068+0000 7f8ac90b2700 1 -- 172.21.15.115:0/661687678 shutdown_connections 2022-09-20T20:06:53.071 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.068+0000 7f8ac90b2700 1 --2- 172.21.15.115:0/661687678 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8ac415c450 0x7f8ac40ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.068+0000 7f8ac90b2700 1 --2- 172.21.15.115:0/661687678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ac40ad5a0 0x7f8ac4151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.068+0000 7f8ac90b2700 1 --2- 172.21.15.115:0/661687678 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8ac415b500 0x7f8ac41522b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.068+0000 7f8ac90b2700 1 -- 172.21.15.115:0/661687678 >> 172.21.15.115:0/661687678 conn(0x7f8ac406a6d0 msgr2=0x7f8ac4154700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.068+0000 7f8ac90b2700 1 -- 172.21.15.115:0/661687678 shutdown_connections 2022-09-20T20:06:53.072 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.068+0000 7f8ac90b2700 1 -- 172.21.15.115:0/661687678 wait complete. 2022-09-20T20:06:53.082 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 8a19aa0f-2885-42c5-a36f-a1dc44bd5489 4 2022-09-20T20:06:53.205 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.203+0000 7f5f27e15700 1 Processor -- start 2022-09-20T20:06:53.206 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.203+0000 7f5f27e15700 1 -- start start 2022-09-20T20:06:53.212 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.204+0000 7f5f27e15700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5f200cf0d0 0x7f5f200c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.204+0000 7f5f27e15700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5f200cf4a0 0x7f5f200cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.204+0000 7f5f27e15700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f200b2640 con 0x7f5f200cc1a0 2022-09-20T20:06:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.204+0000 7f5f27e15700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f200d0080 con 0x7f5f200cf0d0 2022-09-20T20:06:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.204+0000 7f5f27e15700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f200cbec0 con 0x7f5f200cf4a0 2022-09-20T20:06:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.204+0000 7f5f253b0700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5f200cf4a0 0x7f5f200cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.204+0000 7f5f263b2700 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5f200cc1a0 0x7f5f200cc600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48608/0 (socket says 172.21.15.115:48608) 2022-09-20T20:06:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.204+0000 7f5f263b2700 1 -- 172.21.15.115:0/900428503 learned_addr learned my addr 172.21.15.115:0/900428503 (peer_addr_for_me v1:172.21.15.115:0/0) 2022-09-20T20:06:53.215 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.204+0000 7f5f25bb1700 1 --2- 172.21.15.115:0/900428503 >> v2:172.21.15.36:3300/0 conn(0x7f5f200cf0d0 0x7f5f200c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.215 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f253b0700 1 -- 172.21.15.115:0/900428503 >> v2:172.21.15.36:3300/0 conn(0x7f5f200cf0d0 msgr2=0x7f5f200c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:53.215 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f253b0700 1 --2- 172.21.15.115:0/900428503 >> v2:172.21.15.36:3300/0 conn(0x7f5f200cf0d0 0x7f5f200c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.215 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f253b0700 1 -- 172.21.15.115:0/900428503 >> v1:172.21.15.115:6789/0 conn(0x7f5f200cc1a0 legacy=0x7f5f200cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.215 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f253b0700 1 -- 172.21.15.115:0/900428503 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f200cfa30 con 0x7f5f200cf4a0 2022-09-20T20:06:53.216 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f253b0700 1 --2- 172.21.15.115:0/900428503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cf4a0 0x7f5f200cb620 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5f200ccee0 tx=0x7f5f10009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4397733609f086 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:53.216 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f24baf700 1 -- 172.21.15.115:0/900428503 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10011830 con 0x7f5f200cf4a0 2022-09-20T20:06:53.216 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f24baf700 1 -- 172.21.15.115:0/900428503 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f10011990 con 0x7f5f200cf4a0 2022-09-20T20:06:53.216 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f24baf700 1 -- 172.21.15.115:0/900428503 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10011d20 con 0x7f5f200cf4a0 2022-09-20T20:06:53.217 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f27e15700 1 -- 172.21.15.115:0/900428503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cf4a0 msgr2=0x7f5f200cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.217 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f27e15700 1 --2- 172.21.15.115:0/900428503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cf4a0 0x7f5f200cb620 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5f200ccee0 tx=0x7f5f10009b20 comp rx=0 tx=0).stop 2022-09-20T20:06:53.217 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f27e15700 1 -- 172.21.15.115:0/900428503 shutdown_connections 2022-09-20T20:06:53.218 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f27e15700 1 --2- 172.21.15.115:0/900428503 >> v2:172.21.15.36:3300/0 conn(0x7f5f200cf0d0 0x7f5f200c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.218 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f27e15700 1 --2- 172.21.15.115:0/900428503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cf4a0 0x7f5f200cb620 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.218 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.205+0000 7f5f27e15700 1 -- 172.21.15.115:0/900428503 >> 172.21.15.115:0/900428503 conn(0x7f5f2006d190 msgr2=0x7f5f2006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:53.218 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.206+0000 7f5f27e15700 1 -- 172.21.15.115:0/900428503 shutdown_connections 2022-09-20T20:06:53.219 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.206+0000 7f5f27e15700 1 -- 172.21.15.115:0/900428503 wait complete. 2022-09-20T20:06:53.219 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.207+0000 7f5f27e15700 1 Processor -- start 2022-09-20T20:06:53.219 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.207+0000 7f5f27e15700 1 -- start start 2022-09-20T20:06:53.219 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.207+0000 7f5f27e15700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cc1a0 0x7f5f200c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.220 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.207+0000 7f5f27e15700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5f200cf0d0 0x7f5f200a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.220 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.207+0000 7f5f27e15700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f200cf4a0 0x7f5f20158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.220 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.207+0000 7f5f27e15700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200a9010 con 0x7f5f200cf4a0 2022-09-20T20:06:53.221 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.207+0000 7f5f27e15700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200a6a20 con 0x7f5f200cc1a0 2022-09-20T20:06:53.221 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.207+0000 7f5f27e15700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f201592b0 con 0x7f5f200cf0d0 2022-09-20T20:06:53.222 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f25bb1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cc1a0 0x7f5f200c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.222 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f253b0700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5f200cf0d0 0x7f5f200a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.222 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f25bb1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cc1a0 0x7f5f200c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44630/0 (socket says 172.21.15.115:44630) 2022-09-20T20:06:53.223 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f25bb1700 1 -- 172.21.15.115:0/3461090375 learned_addr learned my addr 172.21.15.115:0/3461090375 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:53.223 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f263b2700 1 --2- 172.21.15.115:0/3461090375 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f200cf4a0 0x7f5f20158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.223 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f25bb1700 1 -- 172.21.15.115:0/3461090375 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5f200cf0d0 msgr2=0x7f5f200a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.223 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f25bb1700 1 --2- 172.21.15.115:0/3461090375 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5f200cf0d0 0x7f5f200a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.224 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f25bb1700 1 -- 172.21.15.115:0/3461090375 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f200cf4a0 msgr2=0x7f5f20158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.224 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f25bb1700 1 --2- 172.21.15.115:0/3461090375 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f200cf4a0 0x7f5f20158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.224 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f25bb1700 1 -- 172.21.15.115:0/3461090375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f100094a0 con 0x7f5f200cc1a0 2022-09-20T20:06:53.224 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f253b0700 1 --2- 172.21.15.115:0/3461090375 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5f200cf0d0 0x7f5f200a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-09-20T20:06:53.225 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f25bb1700 1 --2- 172.21.15.115:0/3461090375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cc1a0 0x7f5f200c9280 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00e9b0 tx=0x7f5f1c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:53.225 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f0e7fc700 1 -- 172.21.15.115:0/3461090375 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c009710 con 0x7f5f200cc1a0 2022-09-20T20:06:53.225 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f0e7fc700 1 -- 172.21.15.115:0/3461090375 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00ee50 con 0x7f5f200cc1a0 2022-09-20T20:06:53.226 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f27e15700 1 -- 172.21.15.115:0/3461090375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f200a6c50 con 0x7f5f200cc1a0 2022-09-20T20:06:53.226 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f0e7fc700 1 -- 172.21.15.115:0/3461090375 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00ca50 con 0x7f5f200cc1a0 2022-09-20T20:06:53.226 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f27e15700 1 -- 172.21.15.115:0/3461090375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f200a7210 con 0x7f5f200cc1a0 2022-09-20T20:06:53.226 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f0e7fc700 1 -- 172.21.15.115:0/3461090375 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7f5f1c019030 con 0x7f5f200cc1a0 2022-09-20T20:06:53.227 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.208+0000 7f5f0e7fc700 1 -- 172.21.15.115:0/3461090375 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f5f1c005e30 con 0x7f5f200cc1a0 2022-09-20T20:06:53.227 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.209+0000 7f5f27e15700 1 -- 172.21.15.115:0/3461090375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f201596a0 con 0x7f5f200cc1a0 2022-09-20T20:06:53.228 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:53.210+0000 7f77b518b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-09-20T20:06:53.228 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.213+0000 7f5f0e7fc700 1 -- 172.21.15.115:0/3461090375 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7f5f1c021060 con 0x7f5f200cc1a0 2022-09-20T20:06:53.295 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.294+0000 7f5f27e15700 1 -- 172.21.15.115:0/3461090375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8a19aa0f-2885-42c5-a36f-a1dc44bd5489", "id": 4} v 0) v1 -- 0x7f5f20159490 con 0x7f5f200cc1a0 2022-09-20T20:06:53.299 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.298+0000 7f5f0e7fc700 1 -- 172.21.15.115:0/3461090375 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8a19aa0f-2885-42c5-a36f-a1dc44bd5489", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f1c017070 con 0x7f5f200cc1a0 2022-09-20T20:06:53.300 INFO:teuthology.orchestra.run.smithi115.stdout:4 2022-09-20T20:06:53.301 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.299+0000 7f5f27e15700 1 -- 172.21.15.115:0/3461090375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cc1a0 msgr2=0x7f5f200c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.301 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.299+0000 7f5f27e15700 1 --2- 172.21.15.115:0/3461090375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cc1a0 0x7f5f200c9280 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00e9b0 tx=0x7f5f1c00ecc0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.302 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.299+0000 7f5f27e15700 1 -- 172.21.15.115:0/3461090375 shutdown_connections 2022-09-20T20:06:53.302 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.299+0000 7f5f27e15700 1 --2- 172.21.15.115:0/3461090375 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5f200cf4a0 0x7f5f20158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.302 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.299+0000 7f5f27e15700 1 --2- 172.21.15.115:0/3461090375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f200cc1a0 0x7f5f200c9280 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.302 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.299+0000 7f5f27e15700 1 --2- 172.21.15.115:0/3461090375 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5f200cf0d0 0x7f5f200a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.299+0000 7f5f27e15700 1 -- 172.21.15.115:0/3461090375 >> 172.21.15.115:0/3461090375 conn(0x7f5f2006d190 msgr2=0x7f5f200cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.299+0000 7f5f27e15700 1 -- 172.21.15.115:0/3461090375 shutdown_connections 2022-09-20T20:06:53.303 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.299+0000 7f5f27e15700 1 -- 172.21.15.115:0/3461090375 wait complete. 2022-09-20T20:06:53.312 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new e12282e4-18b8-4bd3-887c-bd913650b1c3 5 2022-09-20T20:06:53.432 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.430+0000 7f8e98d0f700 1 Processor -- start 2022-09-20T20:06:53.432 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.430+0000 7f8e98d0f700 1 -- start start 2022-09-20T20:06:53.438 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.431+0000 7f8e98d0f700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8e940b4fe0 0x7f8e9415da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.439 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.431+0000 7f8e98d0f700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f8e940b1f90 0x7f8e940c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.439 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.431+0000 7f8e98d0f700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e940ca2f0 con 0x7f8e9415cea0 2022-09-20T20:06:53.439 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.431+0000 7f8e98d0f700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e940b53b0 con 0x7f8e940b1f90 2022-09-20T20:06:53.440 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.431+0000 7f8e98d0f700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e94159770 con 0x7f8e940b4fe0 2022-09-20T20:06:53.440 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.431+0000 7f8e9259c700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8e940b4fe0 0x7f8e9415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.440 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.431+0000 7f8e92d9d700 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8e9415cea0 0x7f8e940cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48620/0 (socket says 172.21.15.115:48620) 2022-09-20T20:06:53.440 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.431+0000 7f8e9259c700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8e940b4fe0 0x7f8e9415da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44636/0 (socket says 172.21.15.115:44636) 2022-09-20T20:06:53.440 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.431+0000 7f8e92d9d700 1 -- 172.21.15.115:0/3704236068 learned_addr learned my addr 172.21.15.115:0/3704236068 (peer_addr_for_me v1:172.21.15.115:0/0) 2022-09-20T20:06:53.441 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.431+0000 7f8e91d9b700 1 --2- 172.21.15.115:0/3704236068 >> v2:172.21.15.36:3300/0 conn(0x7f8e940b1f90 0x7f8e940c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.441 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e9259c700 1 -- 172.21.15.115:0/3704236068 >> v2:172.21.15.36:3300/0 conn(0x7f8e940b1f90 msgr2=0x7f8e940c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:53.441 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e9259c700 1 --2- 172.21.15.115:0/3704236068 >> v2:172.21.15.36:3300/0 conn(0x7f8e940b1f90 0x7f8e940c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.441 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e9259c700 1 -- 172.21.15.115:0/3704236068 >> v1:172.21.15.115:6789/0 conn(0x7f8e9415cea0 legacy=0x7f8e940cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.442 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e9259c700 1 -- 172.21.15.115:0/3704236068 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e940ce6b0 con 0x7f8e940b4fe0 2022-09-20T20:06:53.442 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e9159a700 1 -- 172.21.15.115:0/3704236068 <== mon.1 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283001325 0 0) 0x7f8e940ca2f0 con 0x7f8e9415cea0 2022-09-20T20:06:53.442 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e9259c700 1 --2- 172.21.15.115:0/3704236068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e940b4fe0 0x7f8e9415da40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8e940d0860 tx=0x7f8e84009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75a718a983ed9e6c server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:53.442 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e9159a700 1 -- 172.21.15.115:0/3704236068 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84011830 con 0x7f8e940b4fe0 2022-09-20T20:06:53.443 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e9159a700 1 -- 172.21.15.115:0/3704236068 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e84011990 con 0x7f8e940b4fe0 2022-09-20T20:06:53.443 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e9159a700 1 -- 172.21.15.115:0/3704236068 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84011d20 con 0x7f8e940b4fe0 2022-09-20T20:06:53.443 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e98d0f700 1 -- 172.21.15.115:0/3704236068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e940b4fe0 msgr2=0x7f8e9415da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.443 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.432+0000 7f8e98d0f700 1 --2- 172.21.15.115:0/3704236068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e940b4fe0 0x7f8e9415da40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8e940d0860 tx=0x7f8e84009b20 comp rx=0 tx=0).stop 2022-09-20T20:06:53.444 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.433+0000 7f8e98d0f700 1 -- 172.21.15.115:0/3704236068 shutdown_connections 2022-09-20T20:06:53.444 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.433+0000 7f8e98d0f700 1 --2- 172.21.15.115:0/3704236068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e940b4fe0 0x7f8e9415da40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.444 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.433+0000 7f8e98d0f700 1 --2- 172.21.15.115:0/3704236068 >> v2:172.21.15.36:3300/0 conn(0x7f8e940b1f90 0x7f8e940c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.444 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.433+0000 7f8e98d0f700 1 -- 172.21.15.115:0/3704236068 >> 172.21.15.115:0/3704236068 conn(0x7f8e94068ca0 msgr2=0x7f8e940690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:53.445 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.433+0000 7f8e98d0f700 1 -- 172.21.15.115:0/3704236068 shutdown_connections 2022-09-20T20:06:53.445 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.433+0000 7f8e98d0f700 1 -- 172.21.15.115:0/3704236068 wait complete. 2022-09-20T20:06:53.445 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.434+0000 7f8e98d0f700 1 Processor -- start 2022-09-20T20:06:53.445 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.434+0000 7f8e98d0f700 1 -- start start 2022-09-20T20:06:53.445 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e98d0f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e940b1f90 0x7f8e94150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.446 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e98d0f700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8e940b4fe0 0x7f8e941523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.446 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e98d0f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e9415cea0 0x7f8e940c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.446 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e98d0f700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e941545b0 con 0x7f8e940b1f90 2022-09-20T20:06:53.446 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e98d0f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940c43c0 con 0x7f8e9415cea0 2022-09-20T20:06:53.447 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e98d0f700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940c5220 con 0x7f8e940b4fe0 2022-09-20T20:06:53.447 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e92d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e9415cea0 0x7f8e940c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.447 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e9259c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e940b1f90 0x7f8e94150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.447 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e91d9b700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8e940b4fe0 0x7f8e941523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.447 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e92d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e9415cea0 0x7f8e940c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44646/0 (socket says 172.21.15.115:44646) 2022-09-20T20:06:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e92d9d700 1 -- 172.21.15.115:0/2440384037 learned_addr learned my addr 172.21.15.115:0/2440384037 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e92d9d700 1 -- 172.21.15.115:0/2440384037 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8e940b4fe0 msgr2=0x7f8e941523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e92d9d700 1 --2- 172.21.15.115:0/2440384037 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8e940b4fe0 0x7f8e941523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.448 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e92d9d700 1 -- 172.21.15.115:0/2440384037 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e940b1f90 msgr2=0x7f8e94150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.449 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e92d9d700 1 --2- 172.21.15.115:0/2440384037 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e940b1f90 0x7f8e94150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.449 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e92d9d700 1 -- 172.21.15.115:0/2440384037 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e840094a0 con 0x7f8e9415cea0 2022-09-20T20:06:53.449 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e9259c700 1 --2- 172.21.15.115:0/2440384037 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e940b1f90 0x7f8e94150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-09-20T20:06:53.449 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e92d9d700 1 --2- 172.21.15.115:0/2440384037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e9415cea0 0x7f8e940c56b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00cba0 tx=0x7f8e8c00ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:53.449 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e82ffd700 1 -- 172.21.15.115:0/2440384037 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c00aaa0 con 0x7f8e9415cea0 2022-09-20T20:06:53.450 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e82ffd700 1 -- 172.21.15.115:0/2440384037 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8c00ac20 con 0x7f8e9415cea0 2022-09-20T20:06:53.450 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.435+0000 7f8e98d0f700 1 -- 172.21.15.115:0/2440384037 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e940c4610 con 0x7f8e9415cea0 2022-09-20T20:06:53.450 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.436+0000 7f8e82ffd700 1 -- 172.21.15.115:0/2440384037 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c0070d0 con 0x7f8e9415cea0 2022-09-20T20:06:53.450 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.436+0000 7f8e98d0f700 1 -- 172.21.15.115:0/2440384037 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e940c4bd0 con 0x7f8e9415cea0 2022-09-20T20:06:53.451 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.436+0000 7f8e82ffd700 1 -- 172.21.15.115:0/2440384037 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7f8e8c01a030 con 0x7f8e9415cea0 2022-09-20T20:06:53.451 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.436+0000 7f8e82ffd700 1 -- 172.21.15.115:0/2440384037 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f8e8c012970 con 0x7f8e9415cea0 2022-09-20T20:06:53.451 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.437+0000 7f8e98d0f700 1 -- 172.21.15.115:0/2440384037 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e94072aa0 con 0x7f8e9415cea0 2022-09-20T20:06:53.452 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.439+0000 7f8e82ffd700 1 -- 172.21.15.115:0/2440384037 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7f8e8c018070 con 0x7f8e9415cea0 2022-09-20T20:06:53.556 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.555+0000 7f8e98d0f700 1 -- 172.21.15.115:0/2440384037 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e12282e4-18b8-4bd3-887c-bd913650b1c3", "id": 5} v 0) v1 -- 0x7f8e940c5390 con 0x7f8e9415cea0 2022-09-20T20:06:53.560 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.559+0000 7f8e82ffd700 1 -- 172.21.15.115:0/2440384037 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e12282e4-18b8-4bd3-887c-bd913650b1c3", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8e8c012c50 con 0x7f8e9415cea0 2022-09-20T20:06:53.561 INFO:teuthology.orchestra.run.smithi115.stdout:5 2022-09-20T20:06:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.560+0000 7f8e98d0f700 1 -- 172.21.15.115:0/2440384037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e9415cea0 msgr2=0x7f8e940c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.560+0000 7f8e98d0f700 1 --2- 172.21.15.115:0/2440384037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e9415cea0 0x7f8e940c56b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00cba0 tx=0x7f8e8c00ceb0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.565 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.560+0000 7f8e98d0f700 1 -- 172.21.15.115:0/2440384037 shutdown_connections 2022-09-20T20:06:53.565 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.560+0000 7f8e98d0f700 1 --2- 172.21.15.115:0/2440384037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e9415cea0 0x7f8e940c56b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.565 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.560+0000 7f8e98d0f700 1 --2- 172.21.15.115:0/2440384037 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f8e940b1f90 0x7f8e94150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.566 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.560+0000 7f8e98d0f700 1 --2- 172.21.15.115:0/2440384037 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f8e940b4fe0 0x7f8e941523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.566 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.560+0000 7f8e98d0f700 1 -- 172.21.15.115:0/2440384037 >> 172.21.15.115:0/2440384037 conn(0x7f8e94068ca0 msgr2=0x7f8e940b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:53.567 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.560+0000 7f8e98d0f700 1 -- 172.21.15.115:0/2440384037 shutdown_connections 2022-09-20T20:06:53.567 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.560+0000 7f8e98d0f700 1 -- 172.21.15.115:0/2440384037 wait complete. 2022-09-20T20:06:53.574 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 23dd01f7-1bd4-43de-9231-bd81bf351537 6 2022-09-20T20:06:53.644 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:53.643+0000 7f7f713b6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-09-20T20:06:53.695 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.692+0000 7fc1a72c4700 1 Processor -- start 2022-09-20T20:06:53.695 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.693+0000 7fc1a72c4700 1 -- start start 2022-09-20T20:06:53.699 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.693+0000 7fc1a72c4700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fc1a00693b0 0x7fc1a00cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.699 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.693+0000 7fc1a72c4700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1a00af760 0x7fc1a00cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.699 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.693+0000 7fc1a72c4700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a015b640 con 0x7fc1a015b270 2022-09-20T20:06:53.700 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.693+0000 7fc1a72c4700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a00af340 con 0x7fc1a00693b0 2022-09-20T20:06:53.700 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.693+0000 7fc1a72c4700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a00ae470 con 0x7fc1a00af760 2022-09-20T20:06:53.700 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc1a5861700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1a00af760 0x7fc1a00cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.700 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc1a5060700 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc1a015b270 0x7fc1a0068ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48628/0 (socket says 172.21.15.115:48628) 2022-09-20T20:06:53.701 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc1a5060700 1 -- 172.21.15.115:0/1243759897 learned_addr learned my addr 172.21.15.115:0/1243759897 (peer_addr_for_me v1:172.21.15.115:0/0) 2022-09-20T20:06:53.701 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc1a485f700 1 --2- 172.21.15.115:0/1243759897 >> v2:172.21.15.36:3300/0 conn(0x7fc1a00693b0 0x7fc1a00cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.701 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc1a5861700 1 -- 172.21.15.115:0/1243759897 >> v2:172.21.15.36:3300/0 conn(0x7fc1a00693b0 msgr2=0x7fc1a00cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:53.701 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc1a5861700 1 --2- 172.21.15.115:0/1243759897 >> v2:172.21.15.36:3300/0 conn(0x7fc1a00693b0 0x7fc1a00cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.701 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc1a5861700 1 -- 172.21.15.115:0/1243759897 >> v1:172.21.15.115:6789/0 conn(0x7fc1a015b270 legacy=0x7fc1a0068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.702 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc1a5861700 1 -- 172.21.15.115:0/1243759897 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00cf530 con 0x7fc1a00af760 2022-09-20T20:06:53.702 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc18ffff700 1 -- 172.21.15.115:0/1243759897 <== mon.1 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327042742 0 0) 0x7fc1a015b640 con 0x7fc1a015b270 2022-09-20T20:06:53.702 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc1a5861700 1 --2- 172.21.15.115:0/1243759897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00af760 0x7fc1a00cd5a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0069780 tx=0x7fc19800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a0ce2fd402ded server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:53.702 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc18ffff700 1 -- 172.21.15.115:0/1243759897 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19800e070 con 0x7fc1a00af760 2022-09-20T20:06:53.703 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc18ffff700 1 -- 172.21.15.115:0/1243759897 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19800bb20 con 0x7fc1a00af760 2022-09-20T20:06:53.703 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.694+0000 7fc18ffff700 1 -- 172.21.15.115:0/1243759897 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198012430 con 0x7fc1a00af760 2022-09-20T20:06:53.703 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.695+0000 7fc1a72c4700 1 -- 172.21.15.115:0/1243759897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00af760 msgr2=0x7fc1a00cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.703 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.695+0000 7fc1a72c4700 1 --2- 172.21.15.115:0/1243759897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00af760 0x7fc1a00cd5a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0069780 tx=0x7fc19800b330 comp rx=0 tx=0).stop 2022-09-20T20:06:53.704 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.695+0000 7fc1a72c4700 1 -- 172.21.15.115:0/1243759897 shutdown_connections 2022-09-20T20:06:53.704 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.695+0000 7fc1a72c4700 1 --2- 172.21.15.115:0/1243759897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00af760 0x7fc1a00cd5a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.704 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.695+0000 7fc1a72c4700 1 --2- 172.21.15.115:0/1243759897 >> v2:172.21.15.36:3300/0 conn(0x7fc1a00693b0 0x7fc1a00cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.704 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.695+0000 7fc1a72c4700 1 -- 172.21.15.115:0/1243759897 >> 172.21.15.115:0/1243759897 conn(0x7fc1a006a6d0 msgr2=0x7fc1a00ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:53.705 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.695+0000 7fc1a72c4700 1 -- 172.21.15.115:0/1243759897 shutdown_connections 2022-09-20T20:06:53.705 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.695+0000 7fc1a72c4700 1 -- 172.21.15.115:0/1243759897 wait complete. 2022-09-20T20:06:53.705 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.697+0000 7fc1a72c4700 1 Processor -- start 2022-09-20T20:06:53.705 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.697+0000 7fc1a72c4700 1 -- start start 2022-09-20T20:06:53.709 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a72c4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00693b0 0x7fc1a01518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a72c4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1a00af760 0x7fc1a00c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a72c4700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fc1a015b270 0x7fc1a01f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a72c4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a015ba00 con 0x7fc1a00af760 2022-09-20T20:06:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a72c4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a01f4890 con 0x7fc1a00693b0 2022-09-20T20:06:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a72c4700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a015d110 con 0x7fc1a015b270 2022-09-20T20:06:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a5060700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00693b0 0x7fc1a01518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a5861700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fc1a015b270 0x7fc1a01f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a5060700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00693b0 0x7fc1a01518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44654/0 (socket says 172.21.15.115:44654) 2022-09-20T20:06:53.712 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a5060700 1 -- 172.21.15.115:0/418191728 learned_addr learned my addr 172.21.15.115:0/418191728 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:53.712 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.698+0000 7fc1a485f700 1 --2- 172.21.15.115:0/418191728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1a00af760 0x7fc1a00c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.712 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc1a5861700 1 -- 172.21.15.115:0/418191728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00693b0 msgr2=0x7fc1a01518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.712 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc1a5861700 1 --2- 172.21.15.115:0/418191728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00693b0 0x7fc1a01518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc1a5861700 1 -- 172.21.15.115:0/418191728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1a00af760 msgr2=0x7fc1a00c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc1a5861700 1 --2- 172.21.15.115:0/418191728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1a00af760 0x7fc1a00c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc1a5861700 1 -- 172.21.15.115:0/418191728 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198009a70 con 0x7fc1a015b270 2022-09-20T20:06:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc1a5060700 1 --2- 172.21.15.115:0/418191728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00693b0 0x7fc1a01518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-09-20T20:06:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc1a485f700 1 --2- 172.21.15.115:0/418191728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1a00af760 0x7fc1a00c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-09-20T20:06:53.714 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc1a5861700 1 --2- 172.21.15.115:0/418191728 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fc1a015b270 0x7fc1a01f38f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc198016000 tx=0x7fc198004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:53.714 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc18dffb700 1 -- 172.21.15.115:0/418191728 <== mon.2 v2:172.21.15.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198012430 con 0x7fc1a015b270 2022-09-20T20:06:53.714 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc18dffb700 1 -- 172.21.15.115:0/418191728 <== mon.2 v2:172.21.15.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc198020070 con 0x7fc1a015b270 2022-09-20T20:06:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc18dffb700 1 -- 172.21.15.115:0/418191728 <== mon.2 v2:172.21.15.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198004280 con 0x7fc1a015b270 2022-09-20T20:06:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc1a72c4700 1 -- 172.21.15.115:0/418191728 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a01f3ff0 con 0x7fc1a015b270 2022-09-20T20:06:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.699+0000 7fc1a72c4700 1 -- 172.21.15.115:0/418191728 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a01f45b0 con 0x7fc1a015b270 2022-09-20T20:06:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.700+0000 7fc18dffb700 1 -- 172.21.15.115:0/418191728 <== mon.2 v2:172.21.15.115:3301/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7fc19802e070 con 0x7fc1a015b270 2022-09-20T20:06:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.700+0000 7fc1a72c4700 1 -- 172.21.15.115:0/418191728 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a00663c0 con 0x7fc1a015b270 2022-09-20T20:06:53.716 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.700+0000 7fc18dffb700 1 -- 172.21.15.115:0/418191728 <== mon.2 v2:172.21.15.115:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fc198022070 con 0x7fc1a015b270 2022-09-20T20:06:53.716 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.702+0000 7fc18dffb700 1 -- 172.21.15.115:0/418191728 <== mon.2 v2:172.21.15.115:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7fc1980466d0 con 0x7fc1a015b270 2022-09-20T20:06:53.787 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.785+0000 7fc1a72c4700 1 -- 172.21.15.115:0/418191728 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_command({"prefix": "osd new", "uuid": "23dd01f7-1bd4-43de-9231-bd81bf351537", "id": 6} v 0) v1 -- 0x7fc1a00b0db0 con 0x7fc1a015b270 2022-09-20T20:06:53.793 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.792+0000 7fc18dffb700 1 -- 172.21.15.115:0/418191728 <== mon.2 v2:172.21.15.115:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "23dd01f7-1bd4-43de-9231-bd81bf351537", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc198035070 con 0x7fc1a015b270 2022-09-20T20:06:53.794 INFO:teuthology.orchestra.run.smithi115.stdout:6 2022-09-20T20:06:53.797 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.792+0000 7fc1a72c4700 1 -- 172.21.15.115:0/418191728 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fc1a015b270 msgr2=0x7fc1a01f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.797 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.792+0000 7fc1a72c4700 1 --2- 172.21.15.115:0/418191728 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fc1a015b270 0x7fc1a01f38f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc198016000 tx=0x7fc198004ad0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.797 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.793+0000 7fc1a72c4700 1 -- 172.21.15.115:0/418191728 shutdown_connections 2022-09-20T20:06:53.798 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.793+0000 7fc1a72c4700 1 --2- 172.21.15.115:0/418191728 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fc1a015b270 0x7fc1a01f38f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.798 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.793+0000 7fc1a72c4700 1 --2- 172.21.15.115:0/418191728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a00693b0 0x7fc1a01518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.799 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.793+0000 7fc1a72c4700 1 --2- 172.21.15.115:0/418191728 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fc1a00af760 0x7fc1a00c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.799 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.793+0000 7fc1a72c4700 1 -- 172.21.15.115:0/418191728 >> 172.21.15.115:0/418191728 conn(0x7fc1a006a6d0 msgr2=0x7fc1a0159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:53.799 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.793+0000 7fc1a72c4700 1 -- 172.21.15.115:0/418191728 shutdown_connections 2022-09-20T20:06:53.800 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.793+0000 7fc1a72c4700 1 -- 172.21.15.115:0/418191728 wait complete. 2022-09-20T20:06:53.800 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:53.793+0000 7f77b518b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-09-20T20:06:53.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 0414b75c-e580-4955-90ab-adec5f32acbe 7 2022-09-20T20:06:53.809 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:53.806+0000 7f7f713b6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-09-20T20:06:53.929 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.927+0000 7fa60ad86700 1 Processor -- start 2022-09-20T20:06:53.930 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.927+0000 7fa60ad86700 1 -- start start 2022-09-20T20:06:53.933 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.928+0000 7fa60ad86700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fa604070520 0x7fa6040cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.928+0000 7fa60ad86700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa60406f770 0x7fa60406fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.928+0000 7fa60ad86700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60406a640 con 0x7fa6040c98b0 2022-09-20T20:06:53.934 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.928+0000 7fa60ad86700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6040cbec0 con 0x7fa604070520 2022-09-20T20:06:53.935 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.928+0000 7fa60ad86700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60406a110 con 0x7fa60406f770 2022-09-20T20:06:53.935 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.928+0000 7fa609323700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa60406f770 0x7fa60406fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.936 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.928+0000 7fa609323700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa60406f770 0x7fa60406fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44664/0 (socket says 172.21.15.115:44664) 2022-09-20T20:06:53.936 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.928+0000 7fa609323700 1 -- 172.21.15.115:0/19673123 learned_addr learned my addr 172.21.15.115:0/19673123 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:53.937 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.928+0000 7fa603fff700 1 --2- 172.21.15.115:0/19673123 >> v2:172.21.15.36:3300/0 conn(0x7fa604070520 0x7fa6040cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.937 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa609323700 1 -- 172.21.15.115:0/19673123 >> v2:172.21.15.36:3300/0 conn(0x7fa604070520 msgr2=0x7fa6040cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:53.938 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa609323700 1 --2- 172.21.15.115:0/19673123 >> v2:172.21.15.36:3300/0 conn(0x7fa604070520 0x7fa6040cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.938 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa609323700 1 -- 172.21.15.115:0/19673123 >> v1:172.21.15.36:6789/0 conn(0x7fa6040c98b0 legacy=0x7fa604068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.939 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa609323700 1 -- 172.21.15.115:0/19673123 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040700f0 con 0x7fa60406f770 2022-09-20T20:06:53.939 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa609323700 1 --2- 172.21.15.115:0/19673123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60406f770 0x7fa60406fbb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa604064fd0 tx=0x7fa5fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12bd8f54eb244272 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:53.939 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa6037fe700 1 -- 172.21.15.115:0/19673123 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00e070 con 0x7fa60406f770 2022-09-20T20:06:53.940 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa6037fe700 1 -- 172.21.15.115:0/19673123 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc00b910 con 0x7fa60406f770 2022-09-20T20:06:53.940 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa6037fe700 1 -- 172.21.15.115:0/19673123 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00bcc0 con 0x7fa60406f770 2022-09-20T20:06:53.941 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa60ad86700 1 -- 172.21.15.115:0/19673123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60406f770 msgr2=0x7fa60406fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.941 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa60ad86700 1 --2- 172.21.15.115:0/19673123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60406f770 0x7fa60406fbb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa604064fd0 tx=0x7fa5fc00b330 comp rx=0 tx=0).stop 2022-09-20T20:06:53.946 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa60ad86700 1 -- 172.21.15.115:0/19673123 shutdown_connections 2022-09-20T20:06:53.946 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa60ad86700 1 --2- 172.21.15.115:0/19673123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60406f770 0x7fa60406fbb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.947 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa60ad86700 1 --2- 172.21.15.115:0/19673123 >> v2:172.21.15.36:3300/0 conn(0x7fa604070520 0x7fa6040cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.947 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa60ad86700 1 -- 172.21.15.115:0/19673123 >> 172.21.15.115:0/19673123 conn(0x7fa6040b05c0 msgr2=0x7fa6040cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:53.947 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.929+0000 7fa60ad86700 1 -- 172.21.15.115:0/19673123 shutdown_connections 2022-09-20T20:06:53.947 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.930+0000 7fa60ad86700 1 -- 172.21.15.115:0/19673123 wait complete. 2022-09-20T20:06:53.947 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.931+0000 7fa60ad86700 1 Processor -- start 2022-09-20T20:06:53.948 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.931+0000 7fa60ad86700 1 -- start start 2022-09-20T20:06:53.948 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa60ad86700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fa60406f770 0x7fa6041514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.948 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa60ad86700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa604070520 0x7fa6040caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.948 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa60ad86700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040c98b0 0x7fa6040aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:53.949 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa60ad86700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa604153860 con 0x7fa604070520 2022-09-20T20:06:53.949 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa60ad86700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa604152fc0 con 0x7fa6040c98b0 2022-09-20T20:06:53.949 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa60ad86700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6041532a0 con 0x7fa60406f770 2022-09-20T20:06:53.949 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa609323700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040c98b0 0x7fa6040aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.949 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa609323700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040c98b0 0x7fa6040aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44670/0 (socket says 172.21.15.115:44670) 2022-09-20T20:06:53.950 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa609323700 1 -- 172.21.15.115:0/4148480734 learned_addr learned my addr 172.21.15.115:0/4148480734 (peer_addr_for_me v2:172.21.15.115:0/0) 2022-09-20T20:06:53.950 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa608b22700 1 --2- 172.21.15.115:0/4148480734 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fa60406f770 0x7fa6041514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.950 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa603fff700 1 --2- 172.21.15.115:0/4148480734 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa604070520 0x7fa6040caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:53.950 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa609323700 1 -- 172.21.15.115:0/4148480734 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fa60406f770 msgr2=0x7fa6041514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.951 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa609323700 1 --2- 172.21.15.115:0/4148480734 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fa60406f770 0x7fa6041514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.951 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa609323700 1 -- 172.21.15.115:0/4148480734 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa604070520 msgr2=0x7fa6040caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:53.951 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa609323700 1 --2- 172.21.15.115:0/4148480734 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa604070520 0x7fa6040caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:53.951 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa609323700 1 -- 172.21.15.115:0/4148480734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc009b30 con 0x7fa6040c98b0 2022-09-20T20:06:53.951 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa609323700 1 --2- 172.21.15.115:0/4148480734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040c98b0 0x7fa6040aea10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa60406a250 tx=0x7fa5fc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:53.952 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.933+0000 7fa6017fa700 1 -- 172.21.15.115:0/4148480734 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc004880 con 0x7fa6040c98b0 2022-09-20T20:06:53.952 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.934+0000 7fa6017fa700 1 -- 172.21.15.115:0/4148480734 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc0099b0 con 0x7fa6040c98b0 2022-09-20T20:06:53.952 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.934+0000 7fa60ad86700 1 -- 172.21.15.115:0/4148480734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa604153580 con 0x7fa6040c98b0 2022-09-20T20:06:53.952 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.934+0000 7fa6017fa700 1 -- 172.21.15.115:0/4148480734 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc01c680 con 0x7fa6040c98b0 2022-09-20T20:06:53.953 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.934+0000 7fa60ad86700 1 -- 172.21.15.115:0/4148480734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6040caf50 con 0x7fa6040c98b0 2022-09-20T20:06:53.953 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.934+0000 7fa6017fa700 1 -- 172.21.15.115:0/4148480734 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7fa5fc022030 con 0x7fa6040c98b0 2022-09-20T20:06:53.953 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.934+0000 7fa6017fa700 1 -- 172.21.15.115:0/4148480734 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fa5fc025ab0 con 0x7fa6040c98b0 2022-09-20T20:06:53.953 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.935+0000 7fa60ad86700 1 -- 172.21.15.115:0/4148480734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6040cb110 con 0x7fa6040c98b0 2022-09-20T20:06:53.954 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:53.937+0000 7fa6017fa700 1 -- 172.21.15.115:0/4148480734 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7fa5fc020070 con 0x7fa6040c98b0 2022-09-20T20:06:53.955 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:53.954+0000 7f77b518b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-09-20T20:06:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.019+0000 7fa60ad86700 1 -- 172.21.15.115:0/4148480734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0414b75c-e580-4955-90ab-adec5f32acbe", "id": 7} v 0) v1 -- 0x7fa6040ce770 con 0x7fa6040c98b0 2022-09-20T20:06:54.029 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.027+0000 7fa6017fa700 1 -- 172.21.15.115:0/4148480734 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0414b75c-e580-4955-90ab-adec5f32acbe", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa5fc01cc80 con 0x7fa6040c98b0 2022-09-20T20:06:54.030 INFO:teuthology.orchestra.run.smithi115.stdout:7 2022-09-20T20:06:54.031 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.028+0000 7fa60ad86700 1 -- 172.21.15.115:0/4148480734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040c98b0 msgr2=0x7fa6040aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:54.032 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.028+0000 7fa60ad86700 1 --2- 172.21.15.115:0/4148480734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040c98b0 0x7fa6040aea10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa60406a250 tx=0x7fa5fc004ad0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.032 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.028+0000 7fa60ad86700 1 -- 172.21.15.115:0/4148480734 shutdown_connections 2022-09-20T20:06:54.032 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.028+0000 7fa60ad86700 1 --2- 172.21.15.115:0/4148480734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040c98b0 0x7fa6040aea10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.033 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.028+0000 7fa60ad86700 1 --2- 172.21.15.115:0/4148480734 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fa60406f770 0x7fa6041514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.033 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.028+0000 7fa60ad86700 1 --2- 172.21.15.115:0/4148480734 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fa604070520 0x7fa6040caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.033 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.028+0000 7fa60ad86700 1 -- 172.21.15.115:0/4148480734 >> 172.21.15.115:0/4148480734 conn(0x7fa6040b05c0 msgr2=0x7fa60415d580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:54.034 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.028+0000 7fa60ad86700 1 -- 172.21.15.115:0/4148480734 shutdown_connections 2022-09-20T20:06:54.034 INFO:teuthology.orchestra.run.smithi115.stderr:2022-09-20T20:06:54.028+0000 7fa60ad86700 1 -- 172.21.15.115:0/4148480734 wait complete. 2022-09-20T20:06:54.042 INFO:tasks.ceph.osd.0:Restarting daemon 2022-09-20T20:06:54.042 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-09-20T20:06:54.045 INFO:tasks.ceph.osd.0:Started 2022-09-20T20:06:54.045 INFO:tasks.ceph.osd.1:Restarting daemon 2022-09-20T20:06:54.046 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-09-20T20:06:54.048 INFO:tasks.ceph.osd.1:Started 2022-09-20T20:06:54.049 INFO:tasks.ceph.osd.2:Restarting daemon 2022-09-20T20:06:54.049 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-09-20T20:06:54.052 INFO:tasks.ceph.osd.2:Started 2022-09-20T20:06:54.053 INFO:tasks.ceph.osd.3:Restarting daemon 2022-09-20T20:06:54.054 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-09-20T20:06:54.057 INFO:tasks.ceph.osd.3:Started 2022-09-20T20:06:54.058 INFO:tasks.ceph.osd.4:Restarting daemon 2022-09-20T20:06:54.058 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-09-20T20:06:54.061 INFO:tasks.ceph.osd.4:Started 2022-09-20T20:06:54.061 INFO:tasks.ceph.osd.5:Restarting daemon 2022-09-20T20:06:54.062 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-09-20T20:06:54.064 INFO:tasks.ceph.osd.5:Started 2022-09-20T20:06:54.064 INFO:tasks.ceph.osd.6:Restarting daemon 2022-09-20T20:06:54.065 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-09-20T20:06:54.067 INFO:tasks.ceph.osd.6:Started 2022-09-20T20:06:54.067 INFO:tasks.ceph.osd.7:Restarting daemon 2022-09-20T20:06:54.068 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-09-20T20:06:54.070 INFO:tasks.ceph.osd.7:Started 2022-09-20T20:06:54.071 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-09-20T20:06:54.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.212+0000 7fbfdd0e7700 1 Processor -- start 2022-09-20T20:06:54.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.212+0000 7fbfdd0e7700 1 -- start start 2022-09-20T20:06:54.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.213+0000 7fbfdd0e7700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbfd80c8580 0x7fbfd80ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:54.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.213+0000 7fbfdd0e7700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbfd80c8950 0x7fbfd80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:54.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.213+0000 7fbfdd0e7700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd8069bf0 con 0x7fbfd80ab020 2022-09-20T20:06:54.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.213+0000 7fbfdd0e7700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd80aa8f0 con 0x7fbfd80c8580 2022-09-20T20:06:54.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.213+0000 7fbfdd0e7700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd8069e70 con 0x7fbfd80c8950 2022-09-20T20:06:54.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.213+0000 7fbfd6d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbfd80c8580 0x7fbfd80ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:54.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.213+0000 7fbfd6d9d700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fbfd80c8580 0x7fbfd80ad050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:57892/0 (socket says 172.21.15.36:57892) 2022-09-20T20:06:54.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.213+0000 7fbfd6d9d700 1 -- 172.21.15.36:0/2626625557 learned_addr learned my addr 172.21.15.36:0/2626625557 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:06:54.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.213+0000 7fbfd659c700 1 --2- 172.21.15.36:0/2626625557 >> v2:172.21.15.115:3300/0 conn(0x7fbfd80c8950 0x7fbfd80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfd6d9d700 1 -- 172.21.15.36:0/2626625557 >> v2:172.21.15.115:3300/0 conn(0x7fbfd80c8950 msgr2=0x7fbfd80696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfd6d9d700 1 --2- 172.21.15.36:0/2626625557 >> v2:172.21.15.115:3300/0 conn(0x7fbfd80c8950 0x7fbfd80696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfd6d9d700 1 -- 172.21.15.36:0/2626625557 >> v1:172.21.15.115:6789/0 conn(0x7fbfd80ab020 legacy=0x7fbfd80ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfd6d9d700 1 -- 172.21.15.36:0/2626625557 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd806a150 con 0x7fbfd80c8580 2022-09-20T20:06:54.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfd6d9d700 1 --2- 172.21.15.36:0/2626625557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80c8580 0x7fbfd80ad050 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbfd806f290 tx=0x7fbfc8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc5af19efbfac20 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:54.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfd5d9b700 1 -- 172.21.15.36:0/2626625557 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8011830 con 0x7fbfd80c8580 2022-09-20T20:06:54.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfd5d9b700 1 -- 172.21.15.36:0/2626625557 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc8011990 con 0x7fbfd80c8580 2022-09-20T20:06:54.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfd5d9b700 1 -- 172.21.15.36:0/2626625557 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8011d20 con 0x7fbfd80c8580 2022-09-20T20:06:54.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/2626625557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80c8580 msgr2=0x7fbfd80ad050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:54.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfdd0e7700 1 --2- 172.21.15.36:0/2626625557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80c8580 0x7fbfd80ad050 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbfd806f290 tx=0x7fbfc8009bd0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/2626625557 shutdown_connections 2022-09-20T20:06:54.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfdd0e7700 1 --2- 172.21.15.36:0/2626625557 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80c8580 0x7fbfd80ad050 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfdd0e7700 1 --2- 172.21.15.36:0/2626625557 >> v2:172.21.15.115:3300/0 conn(0x7fbfd80c8950 0x7fbfd80696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/2626625557 >> 172.21.15.36:0/2626625557 conn(0x7fbfd80ce950 msgr2=0x7fbfd80ca740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/2626625557 shutdown_connections 2022-09-20T20:06:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.214+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/2626625557 wait complete. 2022-09-20T20:06:54.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.216+0000 7fbfdd0e7700 1 Processor -- start 2022-09-20T20:06:54.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.216+0000 7fbfdd0e7700 1 -- start start 2022-09-20T20:06:54.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfdd0e7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80ab020 0x7fbfd80c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:54.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfdd0e7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfd80c8580 0x7fbfd81e51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:54.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfdd0e7700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fbfd80c8950 0x7fbfd81e5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:54.227 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfdd0e7700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd81f0ee0 con 0x7fbfd80ab020 2022-09-20T20:06:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfdd0e7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd81f11c0 con 0x7fbfd80c8580 2022-09-20T20:06:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfdd0e7700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd81f14a0 con 0x7fbfd80c8950 2022-09-20T20:06:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd6d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80ab020 0x7fbfd80c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd6d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80ab020 0x7fbfd80c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:57896/0 (socket says 172.21.15.36:57896) 2022-09-20T20:06:54.228 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd6d9d700 1 -- 172.21.15.36:0/1560448363 learned_addr learned my addr 172.21.15.36:0/1560448363 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:06:54.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd659c700 1 --2- 172.21.15.36:0/1560448363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfd80c8580 0x7fbfd81e51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:54.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd759e700 1 --2- 172.21.15.36:0/1560448363 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fbfd80c8950 0x7fbfd81e5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:54.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd6d9d700 1 -- 172.21.15.36:0/1560448363 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fbfd80c8950 msgr2=0x7fbfd81e5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:54.229 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd6d9d700 1 --2- 172.21.15.36:0/1560448363 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fbfd80c8950 0x7fbfd81e5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd6d9d700 1 -- 172.21.15.36:0/1560448363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfd80c8580 msgr2=0x7fbfd81e51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:54.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd6d9d700 1 --2- 172.21.15.36:0/1560448363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfd80c8580 0x7fbfd81e51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd6d9d700 1 -- 172.21.15.36:0/1560448363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc8009550 con 0x7fbfd80ab020 2022-09-20T20:06:54.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd659c700 1 --2- 172.21.15.36:0/1560448363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfd80c8580 0x7fbfd81e51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-09-20T20:06:54.230 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfd6d9d700 1 --2- 172.21.15.36:0/1560448363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80ab020 0x7fbfd80c6c00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009c10 tx=0x7fbfc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:54.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.217+0000 7fbfbf7fe700 1 -- 172.21.15.36:0/1560448363 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8004510 con 0x7fbfd80ab020 2022-09-20T20:06:54.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.218+0000 7fbfbf7fe700 1 -- 172.21.15.36:0/1560448363 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc801e070 con 0x7fbfd80ab020 2022-09-20T20:06:54.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.218+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/1560448363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd81f1780 con 0x7fbfd80ab020 2022-09-20T20:06:54.231 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.218+0000 7fbfbf7fe700 1 -- 172.21.15.36:0/1560448363 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80038e0 con 0x7fbfd80ab020 2022-09-20T20:06:54.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.218+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/1560448363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd80c7140 con 0x7fbfd80ab020 2022-09-20T20:06:54.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.218+0000 7fbfbf7fe700 1 -- 172.21.15.36:0/1560448363 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7fbfc800f050 con 0x7fbfd80ab020 2022-09-20T20:06:54.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.218+0000 7fbfbf7fe700 1 -- 172.21.15.36:0/1560448363 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fbfc802dcb0 con 0x7fbfd80ab020 2022-09-20T20:06:54.232 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.218+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/1560448363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd80c9d80 con 0x7fbfd80ab020 2022-09-20T20:06:54.233 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.221+0000 7fbfbf7fe700 1 -- 172.21.15.36:0/1560448363 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7fbfc8020070 con 0x7fbfd80ab020 2022-09-20T20:06:54.292 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.291+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/1560448363 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfd80c4d70 con 0x7fbfd80ab020 2022-09-20T20:06:54.293 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.291+0000 7fbfbf7fe700 1 -- 172.21.15.36:0/1560448363 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fbfc8015020 con 0x7fbfd80ab020 2022-09-20T20:06:54.294 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:06:54.294 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":10,"fsid":"3c118203-7ef2-4e20-8eae-6178b988d685","created":"2022-09-20T20:06:51.484163+0000","modified":"2022-09-20T20:06:54.021671+0000","last_up_change":"0.000000","last_in_change":"2022-09-20T20:06:54.021671+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"027a07c2-8eec-415a-9e2b-ae85e68a47ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"73600a5c-a12c-460c-80ec-6fcd76cc6f3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"76ffb9bc-7fa7-4cdd-8190-8caebf90ccc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dd9de39a-4958-4162-9a85-4d950e22837c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8a19aa0f-2885-42c5-a36f-a1dc44bd5489","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e12282e4-18b8-4bd3-887c-bd913650b1c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"23dd01f7-1bd4-43de-9231-bd81bf351537","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0414b75c-e580-4955-90ab-adec5f32acbe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-20T20:06:54.295 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.292+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/1560448363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80ab020 msgr2=0x7fbfd80c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:54.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.292+0000 7fbfdd0e7700 1 --2- 172.21.15.36:0/1560448363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80ab020 0x7fbfd80c6c00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009c10 tx=0x7fbfc8004750 comp rx=0 tx=0).stop 2022-09-20T20:06:54.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.292+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/1560448363 shutdown_connections 2022-09-20T20:06:54.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.292+0000 7fbfdd0e7700 1 --2- 172.21.15.36:0/1560448363 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fbfd80c8950 0x7fbfd81e5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.296 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.292+0000 7fbfdd0e7700 1 --2- 172.21.15.36:0/1560448363 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fbfd80ab020 0x7fbfd80c6c00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.292+0000 7fbfdd0e7700 1 --2- 172.21.15.36:0/1560448363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfd80c8580 0x7fbfd81e51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:54.297 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.292+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/1560448363 >> 172.21.15.36:0/1560448363 conn(0x7fbfd80ce950 msgr2=0x7fbfd806f590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:54.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.292+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/1560448363 shutdown_connections 2022-09-20T20:06:54.298 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:06:54.292+0000 7fbfdd0e7700 1 -- 172.21.15.36:0/1560448363 wait complete. 2022-09-20T20:06:54.307 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-09-20T20:06:54.307 INFO:tasks.ceph:Waiting for OSDs to come up 2022-09-20T20:06:54.413 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:54.412+0000 7f7f713b6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-09-20T20:06:54.436 INFO:tasks.ceph.osd.1.smithi036.stderr:2022-09-20T20:06:54.434+0000 7f0c590565c0 -1 Falling back to public interface 2022-09-20T20:06:54.595 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:54.594+0000 7f77b518b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-09-20T20:06:54.602 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:54.600+0000 7f7f713b6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-09-20T20:06:54.655 INFO:tasks.ceph.osd.0.smithi036.stderr:2022-09-20T20:06:54.653+0000 7fe3dc9765c0 -1 Falling back to public interface 2022-09-20T20:06:54.681 INFO:tasks.ceph.osd.2.smithi036.stderr:2022-09-20T20:06:54.678+0000 7fc8675835c0 -1 Falling back to public interface 2022-09-20T20:06:54.681 INFO:tasks.ceph.osd.4.smithi115.stderr:2022-09-20T20:06:54.678+0000 7f97895535c0 -1 Falling back to public interface 2022-09-20T20:06:54.682 INFO:tasks.ceph.osd.6.smithi115.stderr:2022-09-20T20:06:54.678+0000 7f507df835c0 -1 Falling back to public interface 2022-09-20T20:06:54.682 INFO:tasks.ceph.osd.6.smithi115.stderr:2022-09-20T20:06:54.678+0000 7f5075547700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2022-09-20T20:06:54.683 INFO:tasks.ceph.osd.6.smithi115.stderr:2022-09-20T20:06:54.678+0000 7f5075547700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2022-09-20T20:06:54.683 INFO:tasks.ceph.osd.5.smithi115.stderr:2022-09-20T20:06:54.682+0000 7f3058a6e5c0 -1 Falling back to public interface 2022-09-20T20:06:54.693 INFO:tasks.ceph.osd.3.smithi036.stderr:2022-09-20T20:06:54.692+0000 7ff27b6fa5c0 -1 Falling back to public interface 2022-09-20T20:06:54.694 INFO:tasks.ceph.osd.7.smithi115.stderr:2022-09-20T20:06:54.692+0000 7f3068cab5c0 -1 Falling back to public interface 2022-09-20T20:06:54.742 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:54.740+0000 7f7f713b6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-09-20T20:06:54.793 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:54.792+0000 7f77b518b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-09-20T20:06:54.898 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:54.897+0000 7f7f713b6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-09-20T20:06:54.935 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:54.933+0000 7f77b518b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-09-20T20:06:55.030 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:55.028+0000 7f7f713b6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-09-20T20:06:55.100 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:55.098+0000 7f77b518b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-09-20T20:06:55.116 DEBUG:teuthology.orchestra.run.smithi036:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-09-20T20:06:55.232 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.230+0000 7f3282d8a700 1 Processor -- start 2022-09-20T20:06:55.233 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.230+0000 7f3282d8a700 1 -- start start 2022-09-20T20:06:55.235 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.231+0000 7f3282d8a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f327c15b2b0 0x7f327c070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:55.236 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.231+0000 7f3282d8a700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f327c068c50 0x7f327c06f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:55.236 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.231+0000 7f3282d8a700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327c15b680 con 0x7f327c15ed30 2022-09-20T20:06:55.236 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.231+0000 7f3282d8a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327c0cc660 con 0x7f327c15b2b0 2022-09-20T20:06:55.236 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.231+0000 7f3282d8a700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327c0cc7f0 con 0x7f327c068c50 2022-09-20T20:06:55.237 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f3280b26700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f327c15b2b0 0x7f327c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:55.237 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f3280b26700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f327c15b2b0 0x7f327c070360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:57902/0 (socket says 172.21.15.36:57902) 2022-09-20T20:06:55.237 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f3280b26700 1 -- 172.21.15.36:0/1298723279 learned_addr learned my addr 172.21.15.36:0/1298723279 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:06:55.237 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f327bfff700 1 --2- 172.21.15.36:0/1298723279 >> v2:172.21.15.115:3300/0 conn(0x7f327c068c50 0x7f327c06f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:55.237 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f3280b26700 1 -- 172.21.15.36:0/1298723279 >> v2:172.21.15.115:3300/0 conn(0x7f327c068c50 msgr2=0x7f327c06f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:06:55.238 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f3280b26700 1 --2- 172.21.15.36:0/1298723279 >> v2:172.21.15.115:3300/0 conn(0x7f327c068c50 0x7f327c06f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:55.238 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f3280b26700 1 -- 172.21.15.36:0/1298723279 >> v1:172.21.15.115:6789/0 conn(0x7f327c15ed30 legacy=0x7f327c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:55.238 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f3280b26700 1 -- 172.21.15.36:0/1298723279 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c15f260 con 0x7f327c15b2b0 2022-09-20T20:06:55.238 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f3280b26700 1 --2- 172.21.15.36:0/1298723279 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c15b2b0 0x7f327c070360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f327c064d00 tx=0x7f3270009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a0d273d6319e2d server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:55.239 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f327b7fe700 1 -- 172.21.15.36:0/1298723279 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270011830 con 0x7f327c15b2b0 2022-09-20T20:06:55.239 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f327b7fe700 1 -- 172.21.15.36:0/1298723279 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3270011990 con 0x7f327c15b2b0 2022-09-20T20:06:55.239 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.232+0000 7f327b7fe700 1 -- 172.21.15.36:0/1298723279 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270011d20 con 0x7f327c15b2b0 2022-09-20T20:06:55.239 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.233+0000 7f3282d8a700 1 -- 172.21.15.36:0/1298723279 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c15b2b0 msgr2=0x7f327c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:55.239 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.233+0000 7f3282d8a700 1 --2- 172.21.15.36:0/1298723279 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c15b2b0 0x7f327c070360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f327c064d00 tx=0x7f3270009bd0 comp rx=0 tx=0).stop 2022-09-20T20:06:55.240 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.233+0000 7f3282d8a700 1 -- 172.21.15.36:0/1298723279 shutdown_connections 2022-09-20T20:06:55.240 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.233+0000 7f3282d8a700 1 --2- 172.21.15.36:0/1298723279 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c15b2b0 0x7f327c070360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:55.240 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.233+0000 7f3282d8a700 1 --2- 172.21.15.36:0/1298723279 >> v2:172.21.15.115:3300/0 conn(0x7f327c068c50 0x7f327c06f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:55.240 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.233+0000 7f3282d8a700 1 -- 172.21.15.36:0/1298723279 >> 172.21.15.36:0/1298723279 conn(0x7f327c0cdce0 msgr2=0x7f327c0ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:55.240 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.233+0000 7f3282d8a700 1 -- 172.21.15.36:0/1298723279 shutdown_connections 2022-09-20T20:06:55.241 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.233+0000 7f3282d8a700 1 -- 172.21.15.36:0/1298723279 wait complete. 2022-09-20T20:06:55.241 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.234+0000 7f3282d8a700 1 Processor -- start 2022-09-20T20:06:55.241 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.234+0000 7f3282d8a700 1 -- start start 2022-09-20T20:06:55.244 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3282d8a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c068c50 0x7f327c0b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:55.244 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3282d8a700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f327c15b2b0 0x7f327c0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:55.245 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3282d8a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f327c15ed30 0x7f327c0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:06:55.245 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3282d8a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c1f1230 con 0x7f327c068c50 2022-09-20T20:06:55.245 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3282d8a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c1f0920 con 0x7f327c15ed30 2022-09-20T20:06:55.245 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3282d8a700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327c1f4cf0 con 0x7f327c15b2b0 2022-09-20T20:06:55.246 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3280b26700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c068c50 0x7f327c0b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:55.246 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3280b26700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c068c50 0x7f327c0b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:57908/0 (socket says 172.21.15.36:57908) 2022-09-20T20:06:55.246 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3280b26700 1 -- 172.21.15.36:0/2551278255 learned_addr learned my addr 172.21.15.36:0/2551278255 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:06:55.246 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3281327700 1 --2- 172.21.15.36:0/2551278255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f327c15ed30 0x7f327c0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:55.246 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f327bfff700 1 --2- 172.21.15.36:0/2551278255 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f327c15b2b0 0x7f327c0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:06:55.247 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3280b26700 1 -- 172.21.15.36:0/2551278255 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f327c15b2b0 msgr2=0x7f327c0b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:55.247 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3280b26700 1 --2- 172.21.15.36:0/2551278255 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f327c15b2b0 0x7f327c0b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:55.247 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3280b26700 1 -- 172.21.15.36:0/2551278255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f327c15ed30 msgr2=0x7f327c0b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:55.247 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.235+0000 7f3280b26700 1 --2- 172.21.15.36:0/2551278255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f327c15ed30 0x7f327c0b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:55.248 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.236+0000 7f3280b26700 1 -- 172.21.15.36:0/2551278255 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3270009550 con 0x7f327c068c50 2022-09-20T20:06:55.248 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.236+0000 7f3280b26700 1 --2- 172.21.15.36:0/2551278255 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c068c50 0x7f327c0b2390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3270000c00 tx=0x7f3270004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:06:55.248 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.236+0000 7f32797fa700 1 -- 172.21.15.36:0/2551278255 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270004380 con 0x7f327c068c50 2022-09-20T20:06:55.248 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.236+0000 7f32797fa700 1 -- 172.21.15.36:0/2551278255 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327001f070 con 0x7f327c068c50 2022-09-20T20:06:55.249 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.236+0000 7f3282d8a700 1 -- 172.21.15.36:0/2551278255 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c1f0be0 con 0x7f327c068c50 2022-09-20T20:06:55.249 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.236+0000 7f32797fa700 1 -- 172.21.15.36:0/2551278255 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32700036a0 con 0x7f327c068c50 2022-09-20T20:06:55.249 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.236+0000 7f3282d8a700 1 -- 172.21.15.36:0/2551278255 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c1f16c0 con 0x7f327c068c50 2022-09-20T20:06:55.249 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.236+0000 7f32797fa700 1 -- 172.21.15.36:0/2551278255 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 1) v1 ==== 517+0+0 (secure 0 0 0) 0x7f327002a020 con 0x7f327c068c50 2022-09-20T20:06:55.249 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.236+0000 7f32797fa700 1 -- 172.21.15.36:0/2551278255 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f327001acb0 con 0x7f327c068c50 2022-09-20T20:06:55.250 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.237+0000 7f3282d8a700 1 -- 172.21.15.36:0/2551278255 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327c072990 con 0x7f327c068c50 2022-09-20T20:06:55.250 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:55.238+0000 7f77b518b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-09-20T20:06:55.251 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.239+0000 7f32797fa700 1 -- 172.21.15.36:0/2551278255 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71346 (secure 0 0 0) 0x7f3270015020 con 0x7f327c068c50 2022-09-20T20:06:55.312 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.310+0000 7f3282d8a700 1 -- 172.21.15.36:0/2551278255 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f327c1f1850 con 0x7f327c068c50 2022-09-20T20:06:55.313 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.311+0000 7f32797fa700 1 -- 172.21.15.36:0/2551278255 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f3270003cb0 con 0x7f327c068c50 2022-09-20T20:06:55.313 INFO:teuthology.misc.health.smithi036.stdout: 2022-09-20T20:06:55.314 INFO:teuthology.misc.health.smithi036.stdout:{"epoch":10,"fsid":"3c118203-7ef2-4e20-8eae-6178b988d685","created":"2022-09-20T20:06:51.484163+0000","modified":"2022-09-20T20:06:54.021671+0000","last_up_change":"0.000000","last_in_change":"2022-09-20T20:06:54.021671+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"027a07c2-8eec-415a-9e2b-ae85e68a47ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"73600a5c-a12c-460c-80ec-6fcd76cc6f3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"76ffb9bc-7fa7-4cdd-8190-8caebf90ccc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dd9de39a-4958-4162-9a85-4d950e22837c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8a19aa0f-2885-42c5-a36f-a1dc44bd5489","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e12282e4-18b8-4bd3-887c-bd913650b1c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"23dd01f7-1bd4-43de-9231-bd81bf351537","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0414b75c-e580-4955-90ab-adec5f32acbe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-20T20:06:55.315 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.312+0000 7f3282d8a700 1 -- 172.21.15.36:0/2551278255 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c068c50 msgr2=0x7f327c0b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:06:55.315 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.312+0000 7f3282d8a700 1 --2- 172.21.15.36:0/2551278255 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c068c50 0x7f327c0b2390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3270000c00 tx=0x7f3270004750 comp rx=0 tx=0).stop 2022-09-20T20:06:55.315 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.312+0000 7f3282d8a700 1 -- 172.21.15.36:0/2551278255 shutdown_connections 2022-09-20T20:06:55.316 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.312+0000 7f3282d8a700 1 --2- 172.21.15.36:0/2551278255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f327c15ed30 0x7f327c0b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:55.316 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.312+0000 7f3282d8a700 1 --2- 172.21.15.36:0/2551278255 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f327c068c50 0x7f327c0b2390 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:55.316 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.312+0000 7f3282d8a700 1 --2- 172.21.15.36:0/2551278255 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f327c15b2b0 0x7f327c0b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:06:55.316 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.312+0000 7f3282d8a700 1 -- 172.21.15.36:0/2551278255 >> 172.21.15.36:0/2551278255 conn(0x7f327c0cdce0 msgr2=0x7f327c0c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:06:55.316 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.312+0000 7f3282d8a700 1 -- 172.21.15.36:0/2551278255 shutdown_connections 2022-09-20T20:06:55.317 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:06:55.312+0000 7f3282d8a700 1 -- 172.21.15.36:0/2551278255 wait complete. 2022-09-20T20:06:55.325 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-09-20T20:06:55.377 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:55.375+0000 7f7f713b6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-09-20T20:06:55.527 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:55.526+0000 7f7f713b6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-09-20T20:06:55.608 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:55.606+0000 7f77b518b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-09-20T20:06:55.639 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:55.638+0000 7f7f713b6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-09-20T20:06:55.766 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:55.764+0000 7f77b518b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-09-20T20:06:55.784 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:55.782+0000 7f7f713b6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-09-20T20:06:55.883 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:55.882+0000 7f7f713b6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-09-20T20:06:55.885 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:55.884+0000 7f77b518b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-09-20T20:06:56.036 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:56.035+0000 7f77b518b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-09-20T20:06:56.139 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:56.137+0000 7f77b518b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-09-20T20:06:56.235 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:56.233+0000 7f7f713b6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-09-20T20:06:56.509 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:56.507+0000 7f77b518b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-09-20T20:06:56.809 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:56.808+0000 7f7f713b6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-09-20T20:06:57.115 INFO:tasks.ceph.mgr.y.smithi036.stderr:2022-09-20T20:06:57.113+0000 7f77b518b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-09-20T20:06:57.837 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:57.836+0000 7f7f2998b700 -1 mgr.server handle_report got status from non-daemon mon.c 2022-09-20T20:06:57.837 INFO:tasks.ceph.mgr.x.smithi115.stderr:2022-09-20T20:06:57.836+0000 7f7f2998b700 -1 mgr.server handle_report got status from non-daemon mon.b 2022-09-20T20:06:57.961 INFO:tasks.ceph.osd.1.smithi036.stderr:2022-09-20T20:06:57.959+0000 7f0c590565c0 -1 osd.1 0 log_to_monitors true 2022-09-20T20:06:57.991 INFO:tasks.ceph.osd.4.smithi115.stderr:2022-09-20T20:06:57.991+0000 7f97895535c0 -1 osd.4 0 log_to_monitors true 2022-09-20T20:06:58.148 INFO:tasks.ceph.osd.0.smithi036.stderr:2022-09-20T20:06:58.145+0000 7fe3dc9765c0 -1 osd.0 0 log_to_monitors true 2022-09-20T20:06:58.184 INFO:tasks.ceph.osd.2.smithi036.stderr:2022-09-20T20:06:58.182+0000 7fc8675835c0 -1 osd.2 0 log_to_monitors true 2022-09-20T20:06:58.190 INFO:tasks.ceph.osd.6.smithi115.stderr:2022-09-20T20:06:58.189+0000 7f507df835c0 -1 osd.6 0 log_to_monitors true 2022-09-20T20:06:58.211 INFO:tasks.ceph.osd.3.smithi036.stderr:2022-09-20T20:06:58.210+0000 7ff27b6fa5c0 -1 osd.3 0 log_to_monitors true 2022-09-20T20:06:58.257 INFO:tasks.ceph.osd.5.smithi115.stderr:2022-09-20T20:06:58.256+0000 7f3058a6e5c0 -1 osd.5 0 log_to_monitors true 2022-09-20T20:06:58.283 INFO:tasks.ceph.osd.7.smithi115.stderr:2022-09-20T20:06:58.283+0000 7f3068cab5c0 -1 osd.7 0 log_to_monitors true 2022-09-20T20:07:01.773 INFO:tasks.ceph.osd.5.smithi115.stderr:2022-09-20T20:07:01.771+0000 7f304c82b700 -1 osd.5 0 waiting for initial osdmap 2022-09-20T20:07:01.788 INFO:tasks.ceph.osd.5.smithi115.stderr:2022-09-20T20:07:01.787+0000 7f3044c06700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-20T20:07:01.830 INFO:tasks.ceph.osd.1.smithi036.stderr:2022-09-20T20:07:01.828+0000 7f0c4ce13700 -1 osd.1 0 waiting for initial osdmap 2022-09-20T20:07:01.856 INFO:tasks.ceph.osd.1.smithi036.stderr:2022-09-20T20:07:01.854+0000 7f0c451ee700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-20T20:07:01.857 INFO:tasks.ceph.osd.6.smithi115.stderr:2022-09-20T20:07:01.854+0000 7f5071d40700 -1 osd.6 0 waiting for initial osdmap 2022-09-20T20:07:01.869 INFO:tasks.ceph.osd.6.smithi115.stderr:2022-09-20T20:07:01.868+0000 7f506a11b700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-20T20:07:01.873 INFO:tasks.ceph.osd.4.smithi115.stderr:2022-09-20T20:07:01.871+0000 7f977d310700 -1 osd.4 0 waiting for initial osdmap 2022-09-20T20:07:01.891 INFO:tasks.ceph.osd.4.smithi115.stderr:2022-09-20T20:07:01.890+0000 7f97756eb700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-20T20:07:01.894 INFO:tasks.ceph.osd.3.smithi036.stderr:2022-09-20T20:07:01.892+0000 7ff26f4b7700 -1 osd.3 0 waiting for initial osdmap 2022-09-20T20:07:01.903 INFO:tasks.ceph.osd.7.smithi115.stderr:2022-09-20T20:07:01.901+0000 7f305ca68700 -1 osd.7 0 waiting for initial osdmap 2022-09-20T20:07:01.908 INFO:tasks.ceph.osd.3.smithi036.stderr:2022-09-20T20:07:01.906+0000 7ff267892700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-20T20:07:01.915 INFO:tasks.ceph.osd.7.smithi115.stderr:2022-09-20T20:07:01.914+0000 7f3054e43700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-20T20:07:01.916 INFO:tasks.ceph.osd.2.smithi036.stderr:2022-09-20T20:07:01.913+0000 7fc85b340700 -1 osd.2 0 waiting for initial osdmap 2022-09-20T20:07:01.931 INFO:tasks.ceph.osd.2.smithi036.stderr:2022-09-20T20:07:01.929+0000 7fc85371b700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-20T20:07:01.932 INFO:tasks.ceph.osd.0.smithi036.stderr:2022-09-20T20:07:01.931+0000 7fe3d0733700 -1 osd.0 0 waiting for initial osdmap 2022-09-20T20:07:01.946 INFO:tasks.ceph.osd.0.smithi036.stderr:2022-09-20T20:07:01.944+0000 7fe3c8b0e700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-20T20:07:02.132 DEBUG:teuthology.orchestra.run.smithi036:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-09-20T20:07:02.248 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.246+0000 7f01d0c45700 1 Processor -- start 2022-09-20T20:07:02.248 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.246+0000 7f01d0c45700 1 -- start start 2022-09-20T20:07:02.252 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.247+0000 7f01d0c45700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc15b300 0x7f01cc0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:02.252 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.247+0000 7f01d0c45700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f01cc0b01f0 0x7f01cc0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:02.252 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.247+0000 7f01d0c45700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01cc0aed50 con 0x7f01cc0d1e80 2022-09-20T20:07:02.253 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.247+0000 7f01d0c45700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01cc0ae460 con 0x7f01cc0b01f0 2022-09-20T20:07:02.253 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.247+0000 7f01d0c45700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0ae700 con 0x7f01cc15b300 2022-09-20T20:07:02.253 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.247+0000 7f01ca59c700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc15b300 0x7f01cc0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:02.253 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.247+0000 7f01cad9d700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f01cc0b01f0 0x7f01cc0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:02.254 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.247+0000 7f01ca59c700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc15b300 0x7f01cc0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3301/0 says I am v2:172.21.15.36:37884/0 (socket says 172.21.15.36:37884) 2022-09-20T20:07:02.254 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.247+0000 7f01ca59c700 1 -- 172.21.15.36:0/2113326196 learned_addr learned my addr 172.21.15.36:0/2113326196 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:02.254 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01c959a700 1 -- 172.21.15.36:0/2113326196 <== mon.1 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784245019 0 0) 0x7f01cc0aed50 con 0x7f01cc0d1e80 2022-09-20T20:07:02.254 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01c959a700 1 -- 172.21.15.36:0/2113326196 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b00035b0 con 0x7f01cc0d1e80 2022-09-20T20:07:02.255 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01cad9d700 1 -- 172.21.15.36:0/2113326196 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc15b300 msgr2=0x7f01cc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:02.255 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01cad9d700 1 --2- 172.21.15.36:0/2113326196 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc15b300 0x7f01cc0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:02.255 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01cad9d700 1 -- 172.21.15.36:0/2113326196 >> v1:172.21.15.115:6789/0 conn(0x7f01cc0d1e80 legacy=0x7f01cc0b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:02.255 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01cad9d700 1 -- 172.21.15.36:0/2113326196 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc0ae9e0 con 0x7f01cc0b01f0 2022-09-20T20:07:02.256 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01ca59c700 1 --2- 172.21.15.36:0/2113326196 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc15b300 0x7f01cc0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-09-20T20:07:02.256 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01cad9d700 1 --2- 172.21.15.36:0/2113326196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01cc0b01f0 0x7f01cc0cf900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009d90 tx=0x7f01c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906e7e2aca82d920 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:02.256 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01c959a700 1 -- 172.21.15.36:0/2113326196 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c400e070 con 0x7f01cc0b01f0 2022-09-20T20:07:02.257 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01c959a700 1 -- 172.21.15.36:0/2113326196 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c400bb20 con 0x7f01cc0b01f0 2022-09-20T20:07:02.257 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.248+0000 7f01c959a700 1 -- 172.21.15.36:0/2113326196 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4012430 con 0x7f01cc0b01f0 2022-09-20T20:07:02.257 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.249+0000 7f01d0c45700 1 -- 172.21.15.36:0/2113326196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01cc0b01f0 msgr2=0x7f01cc0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:02.257 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.249+0000 7f01d0c45700 1 --2- 172.21.15.36:0/2113326196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01cc0b01f0 0x7f01cc0cf900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009d90 tx=0x7f01c400b330 comp rx=0 tx=0).stop 2022-09-20T20:07:02.258 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.249+0000 7f01d0c45700 1 -- 172.21.15.36:0/2113326196 shutdown_connections 2022-09-20T20:07:02.258 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.249+0000 7f01d0c45700 1 --2- 172.21.15.36:0/2113326196 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc15b300 0x7f01cc0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:02.258 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.249+0000 7f01d0c45700 1 --2- 172.21.15.36:0/2113326196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01cc0b01f0 0x7f01cc0cf900 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:02.258 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.249+0000 7f01d0c45700 1 -- 172.21.15.36:0/2113326196 >> 172.21.15.36:0/2113326196 conn(0x7f01cc070ca0 msgr2=0x7f01cc09ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:02.258 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.249+0000 7f01d0c45700 1 -- 172.21.15.36:0/2113326196 shutdown_connections 2022-09-20T20:07:02.259 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.249+0000 7f01d0c45700 1 -- 172.21.15.36:0/2113326196 wait complete. 2022-09-20T20:07:02.259 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.250+0000 7f01d0c45700 1 Processor -- start 2022-09-20T20:07:02.259 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.251+0000 7f01d0c45700 1 -- start start 2022-09-20T20:07:02.262 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.251+0000 7f01d0c45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01cc0d1e80 0x7f01cc09ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:02.262 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.251+0000 7f01d0c45700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc15b300 0x7f01cc1f06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:02.263 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.251+0000 7f01d0c45700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc0b4860 0x7f01cc159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:02.263 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.251+0000 7f01d0c45700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc09f190 con 0x7f01cc15b300 2022-09-20T20:07:02.263 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.251+0000 7f01d0c45700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0b4d60 con 0x7f01cc0d1e80 2022-09-20T20:07:02.263 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01d0c45700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0b5040 con 0x7f01cc0b4860 2022-09-20T20:07:02.264 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01c9d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc15b300 0x7f01cc1f06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:02.264 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01c9d9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc15b300 0x7f01cc1f06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:57972/0 (socket says 172.21.15.36:57972) 2022-09-20T20:07:02.264 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01c9d9b700 1 -- 172.21.15.36:0/765349700 learned_addr learned my addr 172.21.15.36:0/765349700 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:02.265 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01cad9d700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc0b4860 0x7f01cc159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:02.265 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01ca59c700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01cc0d1e80 0x7f01cc09ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:02.265 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01c9d9b700 1 -- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc0b4860 msgr2=0x7f01cc159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:02.265 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01c9d9b700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc0b4860 0x7f01cc159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:02.266 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01c9d9b700 1 -- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01cc0d1e80 msgr2=0x7f01cc09ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:02.266 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01c9d9b700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01cc0d1e80 0x7f01cc09ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:02.266 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01c9d9b700 1 -- 172.21.15.36:0/765349700 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc009550 con 0x7f01cc15b300 2022-09-20T20:07:02.266 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01c9d9b700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc15b300 0x7f01cc1f06b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f01c000c520 tx=0x7f01c000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:02.266 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01baffd700 1 -- 172.21.15.36:0/765349700 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c00052d0 con 0x7f01cc15b300 2022-09-20T20:07:02.267 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01baffd700 1 -- 172.21.15.36:0/765349700 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c0005430 con 0x7f01cc15b300 2022-09-20T20:07:02.267 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01d0c45700 1 -- 172.21.15.36:0/765349700 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c4009a70 con 0x7f01cc15b300 2022-09-20T20:07:02.267 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01baffd700 1 -- 172.21.15.36:0/765349700 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c00038c0 con 0x7f01cc15b300 2022-09-20T20:07:02.267 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.252+0000 7f01d0c45700 1 -- 172.21.15.36:0/765349700 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc15c7e0 con 0x7f01cc15b300 2022-09-20T20:07:02.268 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.253+0000 7f01d0c45700 1 -- 172.21.15.36:0/765349700 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01cc15ce30 con 0x7f01cc15b300 2022-09-20T20:07:02.268 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.254+0000 7f01baffd700 1 -- 172.21.15.36:0/765349700 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 4) v1 ==== 83870+0+0 (secure 0 0 0) 0x7f01c0019030 con 0x7f01cc15b300 2022-09-20T20:07:02.268 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.254+0000 7f01baffd700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f01a8065a00 0x7f01a8067eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:02.268 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.254+0000 7f01baffd700 1 -- 172.21.15.36:0/765349700 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7f01c0080ec0 con 0x7f01cc15b300 2022-09-20T20:07:02.268 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.254+0000 7f01ca59c700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f01a8065a00 0x7f01a8067eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:02.269 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.255+0000 7f01ca59c700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f01a8065a00 0x7f01a8067eb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006d30 tx=0x7f01bc0092c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:02.269 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.259+0000 7f01baffd700 1 -- 172.21.15.36:0/765349700 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f01c004fb80 con 0x7f01cc15b300 2022-09-20T20:07:02.447 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.446+0000 7f01d0c45700 1 -- 172.21.15.36:0/765349700 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01cc072990 con 0x7f01cc15b300 2022-09-20T20:07:02.448 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.446+0000 7f01baffd700 1 -- 172.21.15.36:0/765349700 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f01c0017020 con 0x7f01cc15b300 2022-09-20T20:07:02.449 INFO:teuthology.misc.health.smithi036.stdout: 2022-09-20T20:07:02.449 INFO:teuthology.misc.health.smithi036.stdout:{"epoch":12,"fsid":"3c118203-7ef2-4e20-8eae-6178b988d685","created":"2022-09-20T20:06:51.484163+0000","modified":"2022-09-20T20:06:59.831415+0000","last_up_change":"0.000000","last_in_change":"2022-09-20T20:06:54.021671+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"027a07c2-8eec-415a-9e2b-ae85e68a47ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"73600a5c-a12c-460c-80ec-6fcd76cc6f3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"76ffb9bc-7fa7-4cdd-8190-8caebf90ccc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dd9de39a-4958-4162-9a85-4d950e22837c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8a19aa0f-2885-42c5-a36f-a1dc44bd5489","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e12282e4-18b8-4bd3-887c-bd913650b1c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"23dd01f7-1bd4-43de-9231-bd81bf351537","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0414b75c-e580-4955-90ab-adec5f32acbe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-20T20:07:02.450 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 -- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f01a8065a00 msgr2=0x7f01a8067eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:02.451 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f01a8065a00 0x7f01a8067eb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006d30 tx=0x7f01bc0092c0 comp rx=0 tx=0).stop 2022-09-20T20:07:02.451 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 -- 172.21.15.36:0/765349700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc15b300 msgr2=0x7f01cc1f06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:02.452 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc15b300 0x7f01cc1f06b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f01c000c520 tx=0x7f01c000c830 comp rx=0 tx=0).stop 2022-09-20T20:07:02.452 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 -- 172.21.15.36:0/765349700 shutdown_connections 2022-09-20T20:07:02.452 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f01a8065a00 0x7f01a8067eb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:02.452 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f01cc0b4860 0x7f01cc159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:02.453 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01cc0d1e80 0x7f01cc09ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:02.453 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 --2- 172.21.15.36:0/765349700 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f01cc15b300 0x7f01cc1f06b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:02.453 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 -- 172.21.15.36:0/765349700 >> 172.21.15.36:0/765349700 conn(0x7f01cc070ca0 msgr2=0x7f01cc0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:02.453 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 -- 172.21.15.36:0/765349700 shutdown_connections 2022-09-20T20:07:02.453 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:02.448+0000 7f01d0c45700 1 -- 172.21.15.36:0/765349700 wait complete. 2022-09-20T20:07:02.461 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-09-20T20:07:09.269 DEBUG:teuthology.orchestra.run.smithi036:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-09-20T20:07:09.398 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.397+0000 7fcd5f39a700 1 Processor -- start 2022-09-20T20:07:09.399 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.397+0000 7fcd5f39a700 1 -- start start 2022-09-20T20:07:09.401 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5f39a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd58157bc0 0x7fcd5815b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.402 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5f39a700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5815bbe0 con 0x7fcd58069360 2022-09-20T20:07:09.402 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5f39a700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5815bec0 con 0x7fcd581577f0 2022-09-20T20:07:09.402 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5f39a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd580cb9e0 con 0x7fcd58157bc0 2022-09-20T20:07:09.402 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5d937700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd58157bc0 0x7fcd5815b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.403 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5c935700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7fcd58069360 0x7fcd580cbe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:37594/0 (socket says 172.21.15.36:37594) 2022-09-20T20:07:09.403 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5d937700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7fcd58157bc0 0x7fcd5815b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58158/0 (socket says 172.21.15.36:58158) 2022-09-20T20:07:09.403 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5c935700 1 -- 172.21.15.36:0/4203981369 learned_addr learned my addr 172.21.15.36:0/4203981369 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-09-20T20:07:09.403 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5d937700 1 -- 172.21.15.36:0/4203981369 >> v1:172.21.15.115:6789/0 conn(0x7fcd581577f0 legacy=0x7fcd58068c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.404 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5d937700 1 -- 172.21.15.36:0/4203981369 >> v1:172.21.15.36:6789/0 conn(0x7fcd58069360 legacy=0x7fcd580cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.404 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5d937700 1 -- 172.21.15.36:0/4203981369 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5815c1a0 con 0x7fcd58157bc0 2022-09-20T20:07:09.404 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd47fff700 1 -- 172.21.15.36:0/4203981369 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835031653 0 0) 0x7fcd5815bbe0 con 0x7fcd58069360 2022-09-20T20:07:09.404 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.398+0000 7fcd5d937700 1 --2- 172.21.15.36:0/4203981369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd58157bc0 0x7fcd5815b4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009d90 tx=0x7fcd5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597c040753ea630e server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:09.404 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.399+0000 7fcd47fff700 1 -- 172.21.15.36:0/4203981369 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000e070 con 0x7fcd58157bc0 2022-09-20T20:07:09.405 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.399+0000 7fcd47fff700 1 -- 172.21.15.36:0/4203981369 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5000bbd0 con 0x7fcd58157bc0 2022-09-20T20:07:09.405 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.399+0000 7fcd47fff700 1 -- 172.21.15.36:0/4203981369 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50012430 con 0x7fcd58157bc0 2022-09-20T20:07:09.405 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.399+0000 7fcd5f39a700 1 -- 172.21.15.36:0/4203981369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd58157bc0 msgr2=0x7fcd5815b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.405 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.399+0000 7fcd5f39a700 1 --2- 172.21.15.36:0/4203981369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd58157bc0 0x7fcd5815b4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009d90 tx=0x7fcd5000b330 comp rx=0 tx=0).stop 2022-09-20T20:07:09.406 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.399+0000 7fcd5f39a700 1 -- 172.21.15.36:0/4203981369 shutdown_connections 2022-09-20T20:07:09.406 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.399+0000 7fcd5f39a700 1 --2- 172.21.15.36:0/4203981369 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd58157bc0 0x7fcd5815b4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.407 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.399+0000 7fcd5f39a700 1 -- 172.21.15.36:0/4203981369 >> 172.21.15.36:0/4203981369 conn(0x7fcd580afbe0 msgr2=0x7fcd580c9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:09.407 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.399+0000 7fcd5f39a700 1 -- 172.21.15.36:0/4203981369 shutdown_connections 2022-09-20T20:07:09.407 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.399+0000 7fcd5f39a700 1 -- 172.21.15.36:0/4203981369 wait complete. 2022-09-20T20:07:09.411 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.400+0000 7fcd5f39a700 1 Processor -- start 2022-09-20T20:07:09.412 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.401+0000 7fcd5f39a700 1 -- start start 2022-09-20T20:07:09.412 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.401+0000 7fcd5f39a700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcd58069360 0x7fcd5815ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.412 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.401+0000 7fcd5f39a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd581577f0 0x7fcd580b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.413 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.401+0000 7fcd5f39a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd58157bc0 0x7fcd5814a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.413 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.401+0000 7fcd5f39a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58157fc0 con 0x7fcd581577f0 2022-09-20T20:07:09.413 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.401+0000 7fcd5f39a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580b2a80 con 0x7fcd58157bc0 2022-09-20T20:07:09.414 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.401+0000 7fcd5f39a700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580b2d60 con 0x7fcd58069360 2022-09-20T20:07:09.414 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.401+0000 7fcd5c935700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd581577f0 0x7fcd580b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.415 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.401+0000 7fcd5c935700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd581577f0 0x7fcd580b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58162/0 (socket says 172.21.15.36:58162) 2022-09-20T20:07:09.415 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.401+0000 7fcd5c935700 1 -- 172.21.15.36:0/3726907884 learned_addr learned my addr 172.21.15.36:0/3726907884 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:09.415 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5d937700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd58157bc0 0x7fcd5814a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.416 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5d136700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcd58069360 0x7fcd5815ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.416 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5c935700 1 -- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcd58069360 msgr2=0x7fcd5815ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.417 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5c935700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcd58069360 0x7fcd5815ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.417 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5c935700 1 -- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd58157bc0 msgr2=0x7fcd5814a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.417 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5c935700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd58157bc0 0x7fcd5814a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.418 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5c935700 1 -- 172.21.15.36:0/3726907884 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009a70 con 0x7fcd581577f0 2022-09-20T20:07:09.418 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5d937700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd58157bc0 0x7fcd5814a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-09-20T20:07:09.418 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5c935700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd581577f0 0x7fcd580b21a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcd48001eb0 tx=0x7fcd4800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:09.418 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd45ffb700 1 -- 172.21.15.36:0/3726907884 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48004c80 con 0x7fcd581577f0 2022-09-20T20:07:09.419 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd45ffb700 1 -- 172.21.15.36:0/3726907884 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd48004de0 con 0x7fcd581577f0 2022-09-20T20:07:09.419 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd45ffb700 1 -- 172.21.15.36:0/3726907884 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48007310 con 0x7fcd581577f0 2022-09-20T20:07:09.419 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5f39a700 1 -- 172.21.15.36:0/3726907884 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd580b3040 con 0x7fcd581577f0 2022-09-20T20:07:09.419 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.402+0000 7fcd5f39a700 1 -- 172.21.15.36:0/3726907884 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd580b26e0 con 0x7fcd581577f0 2022-09-20T20:07:09.420 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.403+0000 7fcd5f39a700 1 -- 172.21.15.36:0/3726907884 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd580664c0 con 0x7fcd581577f0 2022-09-20T20:07:09.420 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.403+0000 7fcd45ffb700 1 -- 172.21.15.36:0/3726907884 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7fcd4801a020 con 0x7fcd581577f0 2022-09-20T20:07:09.420 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.404+0000 7fcd45ffb700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcd38065a50 0x7fcd38067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.420 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.404+0000 7fcd45ffb700 1 -- 172.21.15.36:0/3726907884 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcd480821d0 con 0x7fcd581577f0 2022-09-20T20:07:09.421 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.404+0000 7fcd5d136700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcd38065a50 0x7fcd38067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.421 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.405+0000 7fcd5d136700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcd38065a50 0x7fcd38067f00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd54007fd0 tx=0x7fcd54009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:09.421 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.409+0000 7fcd45ffb700 1 -- 172.21.15.36:0/3726907884 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7fcd480505a0 con 0x7fcd581577f0 2022-09-20T20:07:09.603 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.601+0000 7fcd5f39a700 1 -- 172.21.15.36:0/3726907884 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd58072990 con 0x7fcd581577f0 2022-09-20T20:07:09.604 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.602+0000 7fcd45ffb700 1 -- 172.21.15.36:0/3726907884 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12175 (secure 0 0 0) 0x7fcd48053bc0 con 0x7fcd581577f0 2022-09-20T20:07:09.604 INFO:teuthology.misc.health.smithi036.stdout: 2022-09-20T20:07:09.605 INFO:teuthology.misc.health.smithi036.stdout:{"epoch":16,"fsid":"3c118203-7ef2-4e20-8eae-6178b988d685","created":"2022-09-20T20:06:51.484163+0000","modified":"2022-09-20T20:07:05.855258+0000","last_up_change":"2022-09-20T20:07:02.800559+0000","last_in_change":"2022-09-20T20:06:54.021671+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-09-20T20:07:02.886289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"027a07c2-8eec-415a-9e2b-ae85e68a47ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":99855},{"type":"v1","addr":"172.21.15.36:6809","nonce":99855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":99855},{"type":"v1","addr":"172.21.15.36:6811","nonce":99855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":99855},{"type":"v1","addr":"172.21.15.36:6815","nonce":99855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":99855},{"type":"v1","addr":"172.21.15.36:6813","nonce":99855}]},"public_addr":"172.21.15.36:6809/99855","cluster_addr":"172.21.15.36:6811/99855","heartbeat_back_addr":"172.21.15.36:6815/99855","heartbeat_front_addr":"172.21.15.36:6813/99855","state":["exists","up"]},{"osd":1,"uuid":"73600a5c-a12c-460c-80ec-6fcd76cc6f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6800","nonce":99858},{"type":"v1","addr":"172.21.15.36:6801","nonce":99858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":99858},{"type":"v1","addr":"172.21.15.36:6803","nonce":99858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":99858},{"type":"v1","addr":"172.21.15.36:6807","nonce":99858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":99858},{"type":"v1","addr":"172.21.15.36:6805","nonce":99858}]},"public_addr":"172.21.15.36:6801/99858","cluster_addr":"172.21.15.36:6803/99858","heartbeat_back_addr":"172.21.15.36:6807/99858","heartbeat_front_addr":"172.21.15.36:6805/99858","state":["exists","up"]},{"osd":2,"uuid":"76ffb9bc-7fa7-4cdd-8190-8caebf90ccc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":99871},{"type":"v1","addr":"172.21.15.36:6817","nonce":99871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":99871},{"type":"v1","addr":"172.21.15.36:6819","nonce":99871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":99871},{"type":"v1","addr":"172.21.15.36:6823","nonce":99871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":99871},{"type":"v1","addr":"172.21.15.36:6821","nonce":99871}]},"public_addr":"172.21.15.36:6817/99871","cluster_addr":"172.21.15.36:6819/99871","heartbeat_back_addr":"172.21.15.36:6823/99871","heartbeat_front_addr":"172.21.15.36:6821/99871","state":["exists","up"]},{"osd":3,"uuid":"dd9de39a-4958-4162-9a85-4d950e22837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":99872},{"type":"v1","addr":"172.21.15.36:6825","nonce":99872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":99872},{"type":"v1","addr":"172.21.15.36:6827","nonce":99872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":99872},{"type":"v1","addr":"172.21.15.36:6831","nonce":99872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":99872},{"type":"v1","addr":"172.21.15.36:6829","nonce":99872}]},"public_addr":"172.21.15.36:6825/99872","cluster_addr":"172.21.15.36:6827/99872","heartbeat_back_addr":"172.21.15.36:6831/99872","heartbeat_front_addr":"172.21.15.36:6829/99872","state":["exists","up"]},{"osd":4,"uuid":"8a19aa0f-2885-42c5-a36f-a1dc44bd5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":100090},{"type":"v1","addr":"172.21.15.115:6801","nonce":100090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":100090},{"type":"v1","addr":"172.21.15.115:6805","nonce":100090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":100090},{"type":"v1","addr":"172.21.15.115:6813","nonce":100090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":100090},{"type":"v1","addr":"172.21.15.115:6809","nonce":100090}]},"public_addr":"172.21.15.115:6801/100090","cluster_addr":"172.21.15.115:6805/100090","heartbeat_back_addr":"172.21.15.115:6813/100090","heartbeat_front_addr":"172.21.15.115:6809/100090","state":["exists","up"]},{"osd":5,"uuid":"e12282e4-18b8-4bd3-887c-bd913650b1c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":100091},{"type":"v1","addr":"172.21.15.115:6817","nonce":100091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":100091},{"type":"v1","addr":"172.21.15.115:6819","nonce":100091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":100091},{"type":"v1","addr":"172.21.15.115:6823","nonce":100091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":100091},{"type":"v1","addr":"172.21.15.115:6821","nonce":100091}]},"public_addr":"172.21.15.115:6817/100091","cluster_addr":"172.21.15.115:6819/100091","heartbeat_back_addr":"172.21.15.115:6823/100091","heartbeat_front_addr":"172.21.15.115:6821/100091","state":["exists","up"]},{"osd":6,"uuid":"23dd01f7-1bd4-43de-9231-bd81bf351537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":100092},{"type":"v1","addr":"172.21.15.115:6804","nonce":100092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":100092},{"type":"v1","addr":"172.21.15.115:6808","nonce":100092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":100092},{"type":"v1","addr":"172.21.15.115:6815","nonce":100092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":100092},{"type":"v1","addr":"172.21.15.115:6812","nonce":100092}]},"public_addr":"172.21.15.115:6804/100092","cluster_addr":"172.21.15.115:6808/100092","heartbeat_back_addr":"172.21.15.115:6815/100092","heartbeat_front_addr":"172.21.15.115:6812/100092","state":["exists","up"]},{"osd":7,"uuid":"0414b75c-e580-4955-90ab-adec5f32acbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":100135},{"type":"v1","addr":"172.21.15.115:6825","nonce":100135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":100135},{"type":"v1","addr":"172.21.15.115:6827","nonce":100135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":100135},{"type":"v1","addr":"172.21.15.115:6831","nonce":100135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":100135},{"type":"v1","addr":"172.21.15.115:6829","nonce":100135}]},"public_addr":"172.21.15.115:6825/100135","cluster_addr":"172.21.15.115:6827/100135","heartbeat_back_addr":"172.21.15.115:6831/100135","heartbeat_front_addr":"172.21.15.115:6829/100135","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.122892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:58.979821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.136461+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.220479+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:58.971449+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.275615+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.200553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.332596+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-20T20:07:09.606 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 -- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcd38065a50 msgr2=0x7fcd38067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.607 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcd38065a50 0x7fcd38067f00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd54007fd0 tx=0x7fcd54009040 comp rx=0 tx=0).stop 2022-09-20T20:07:09.607 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 -- 172.21.15.36:0/3726907884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd581577f0 msgr2=0x7fcd580b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.607 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd581577f0 0x7fcd580b21a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcd48001eb0 tx=0x7fcd4800c8b0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.608 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 -- 172.21.15.36:0/3726907884 shutdown_connections 2022-09-20T20:07:09.608 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcd38065a50 0x7fcd38067f00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.608 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd58157bc0 0x7fcd5814a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.608 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcd58069360 0x7fcd5815ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.608 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 --2- 172.21.15.36:0/3726907884 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcd581577f0 0x7fcd580b21a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.609 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 -- 172.21.15.36:0/3726907884 >> 172.21.15.36:0/3726907884 conn(0x7fcd580afbe0 msgr2=0x7fcd580ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:09.609 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 -- 172.21.15.36:0/3726907884 shutdown_connections 2022-09-20T20:07:09.609 INFO:teuthology.misc.health.smithi036.stderr:2022-09-20T20:07:09.604+0000 7fcd5f39a700 1 -- 172.21.15.36:0/3726907884 wait complete. 2022-09-20T20:07:09.617 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-09-20T20:07:09.618 INFO:tasks.ceph:Creating RBD pool 2022-09-20T20:07:09.618 DEBUG:teuthology.orchestra.run.smithi036:> sudo ceph --cluster ceph osd pool create rbd 8 2022-09-20T20:07:09.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.740+0000 7fceec42e700 1 Processor -- start 2022-09-20T20:07:09.742 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.740+0000 7fceec42e700 1 -- start start 2022-09-20T20:07:09.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.741+0000 7fceec42e700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee415b220 0x7fcee40aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.741+0000 7fceec42e700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcee40cc730 0x7fcee40cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.741+0000 7fceec42e700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee415c600 con 0x7fcee40c9e40 2022-09-20T20:07:09.746 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.741+0000 7fceec42e700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee40af110 con 0x7fcee40cc730 2022-09-20T20:07:09.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.741+0000 7fceec42e700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40cee80 con 0x7fcee415b220 2022-09-20T20:07:09.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.741+0000 7fceea9cb700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcee40cc730 0x7fcee40cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.741+0000 7fceea1ca700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee415b220 0x7fcee40aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.741+0000 7fceea9cb700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcee40cc730 0x7fcee40cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.36:51786/0 (socket says 172.21.15.36:51786) 2022-09-20T20:07:09.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.741+0000 7fceea9cb700 1 -- 172.21.15.36:0/3533045746 learned_addr learned my addr 172.21.15.36:0/3533045746 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:09.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.741+0000 7fcee99c9700 1 --1- 172.21.15.36:0/3533045746 >> v1:172.21.15.115:6789/0 conn(0x7fcee40c9e40 0x7fcee40cc020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.36:48242/0 (socket says 172.21.15.36:48242) 2022-09-20T20:07:09.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.742+0000 7fceea1ca700 1 -- 172.21.15.36:0/3533045746 >> v2:172.21.15.115:3300/0 conn(0x7fcee40cc730 msgr2=0x7fcee40cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:09.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.742+0000 7fceea1ca700 1 --2- 172.21.15.36:0/3533045746 >> v2:172.21.15.115:3300/0 conn(0x7fcee40cc730 0x7fcee40cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.742+0000 7fceea1ca700 1 -- 172.21.15.36:0/3533045746 >> v1:172.21.15.115:6789/0 conn(0x7fcee40c9e40 legacy=0x7fcee40cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.742+0000 7fceea1ca700 1 -- 172.21.15.36:0/3533045746 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40cf160 con 0x7fcee415b220 2022-09-20T20:07:09.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.742+0000 7fceea1ca700 1 --2- 172.21.15.36:0/3533045746 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee415b220 0x7fcee40aebd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fced4009840 tx=0x7fced4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2c60e2f26857edd server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:09.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.742+0000 7fcee91c8700 1 -- 172.21.15.36:0/3533045746 <== mon.2 v2:172.21.15.115:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4011830 con 0x7fcee415b220 2022-09-20T20:07:09.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.742+0000 7fcee91c8700 1 -- 172.21.15.36:0/3533045746 <== mon.2 v2:172.21.15.115:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced4011990 con 0x7fcee415b220 2022-09-20T20:07:09.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.742+0000 7fcee91c8700 1 -- 172.21.15.36:0/3533045746 <== mon.2 v2:172.21.15.115:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4011d20 con 0x7fcee415b220 2022-09-20T20:07:09.750 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.742+0000 7fceec42e700 1 -- 172.21.15.36:0/3533045746 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee415b220 msgr2=0x7fcee40aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.743+0000 7fceec42e700 1 --2- 172.21.15.36:0/3533045746 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee415b220 0x7fcee40aebd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fced4009840 tx=0x7fced4009b50 comp rx=0 tx=0).stop 2022-09-20T20:07:09.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.743+0000 7fceec42e700 1 -- 172.21.15.36:0/3533045746 shutdown_connections 2022-09-20T20:07:09.751 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.743+0000 7fceec42e700 1 --2- 172.21.15.36:0/3533045746 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee415b220 0x7fcee40aebd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.743+0000 7fceec42e700 1 --2- 172.21.15.36:0/3533045746 >> v2:172.21.15.115:3300/0 conn(0x7fcee40cc730 0x7fcee40cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.743+0000 7fceec42e700 1 -- 172.21.15.36:0/3533045746 >> 172.21.15.36:0/3533045746 conn(0x7fcee406d190 msgr2=0x7fcee406d560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:09.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.743+0000 7fceec42e700 1 -- 172.21.15.36:0/3533045746 shutdown_connections 2022-09-20T20:07:09.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.743+0000 7fceec42e700 1 -- 172.21.15.36:0/3533045746 wait complete. 2022-09-20T20:07:09.752 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.744+0000 7fceec42e700 1 Processor -- start 2022-09-20T20:07:09.753 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.744+0000 7fceec42e700 1 -- start start 2022-09-20T20:07:09.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.745+0000 7fceec42e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcee40c9e40 0x7fcee4159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.745+0000 7fceec42e700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee40cc730 0x7fcee415c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.757 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.745+0000 7fceec42e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcee415b220 0x7fcee415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.745+0000 7fceec42e700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee4159d60 con 0x7fcee40c9e40 2022-09-20T20:07:09.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.745+0000 7fceec42e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee41f1c10 con 0x7fcee415b220 2022-09-20T20:07:09.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.745+0000 7fceec42e700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee41f1090 con 0x7fcee40cc730 2022-09-20T20:07:09.758 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.745+0000 7fceea1ca700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcee40c9e40 0x7fcee4159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.745+0000 7fceea1ca700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcee40c9e40 0x7fcee4159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58172/0 (socket says 172.21.15.36:58172) 2022-09-20T20:07:09.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.745+0000 7fceea1ca700 1 -- 172.21.15.36:0/760272921 learned_addr learned my addr 172.21.15.36:0/760272921 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:09.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fceea9cb700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcee415b220 0x7fcee415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.759 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fcee99c9700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee40cc730 0x7fcee415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fceea1ca700 1 -- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee40cc730 msgr2=0x7fcee415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fceea1ca700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee40cc730 0x7fcee415c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fceea1ca700 1 -- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcee415b220 msgr2=0x7fcee415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:09.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fceea1ca700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcee415b220 0x7fcee415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:09.760 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fceea1ca700 1 -- 172.21.15.36:0/760272921 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced40094a0 con 0x7fcee40c9e40 2022-09-20T20:07:09.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fceea9cb700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcee415b220 0x7fcee415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-09-20T20:07:09.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fceea1ca700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcee40c9e40 0x7fcee4159820 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fced4014040 tx=0x7fced4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:09.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fced2ffd700 1 -- 172.21.15.36:0/760272921 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced40038a0 con 0x7fcee40c9e40 2022-09-20T20:07:09.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fced2ffd700 1 -- 172.21.15.36:0/760272921 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced401f070 con 0x7fcee40c9e40 2022-09-20T20:07:09.761 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fceec42e700 1 -- 172.21.15.36:0/760272921 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee41f1370 con 0x7fcee40c9e40 2022-09-20T20:07:09.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fced2ffd700 1 -- 172.21.15.36:0/760272921 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4003c00 con 0x7fcee40c9e40 2022-09-20T20:07:09.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.746+0000 7fceec42e700 1 -- 172.21.15.36:0/760272921 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee41f1930 con 0x7fcee40c9e40 2022-09-20T20:07:09.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.747+0000 7fceec42e700 1 -- 172.21.15.36:0/760272921 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee41f1ed0 con 0x7fcee40c9e40 2022-09-20T20:07:09.762 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.747+0000 7fced2ffd700 1 -- 172.21.15.36:0/760272921 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7fced400f050 con 0x7fcee40c9e40 2022-09-20T20:07:09.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.748+0000 7fced2ffd700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcec4065a50 0x7fcec4067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:09.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.748+0000 7fced2ffd700 1 -- 172.21.15.36:0/760272921 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fced4021070 con 0x7fcee40c9e40 2022-09-20T20:07:09.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.748+0000 7fcee99c9700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcec4065a50 0x7fcec4067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:09.763 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.748+0000 7fcee99c9700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcec4065a50 0x7fcec4067f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcee415a110 tx=0x7fcee0007d10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:09.764 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.752+0000 7fced2ffd700 1 -- 172.21.15.36:0/760272921 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7fcee41f1ed0 con 0x7fcee40c9e40 2022-09-20T20:07:09.941 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:09.939+0000 7fceec42e700 1 -- 172.21.15.36:0/760272921 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fcee41f1ed0 con 0x7fcee40c9e40 2022-09-20T20:07:10.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.254+0000 7fced2ffd700 1 -- 172.21.15.36:0/760272921 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fced402b0a0 con 0x7fcee40c9e40 2022-09-20T20:07:10.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 -- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcec4065a50 msgr2=0x7fcec4067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:10.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcec4065a50 0x7fcec4067f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcee415a110 tx=0x7fcee0007d10 comp rx=0 tx=0).stop 2022-09-20T20:07:10.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 -- 172.21.15.36:0/760272921 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcee40c9e40 msgr2=0x7fcee4159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:10.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcee40c9e40 0x7fcee4159820 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fced4014040 tx=0x7fced4004750 comp rx=0 tx=0).stop 2022-09-20T20:07:10.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 -- 172.21.15.36:0/760272921 shutdown_connections 2022-09-20T20:07:10.260 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fcec4065a50 0x7fcec4067f00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:10.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcee415b220 0x7fcee415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:10.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fcee40c9e40 0x7fcee4159820 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:10.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 --2- 172.21.15.36:0/760272921 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fcee40cc730 0x7fcee415c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:10.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 -- 172.21.15.36:0/760272921 >> 172.21.15.36:0/760272921 conn(0x7fcee406d190 msgr2=0x7fcee40cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:10.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.257+0000 7fceec42e700 1 -- 172.21.15.36:0/760272921 shutdown_connections 2022-09-20T20:07:10.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.258+0000 7fceec42e700 1 -- 172.21.15.36:0/760272921 wait complete. 2022-09-20T20:07:10.262 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'rbd' created 2022-09-20T20:07:10.273 DEBUG:teuthology.orchestra.run.smithi036:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-09-20T20:07:10.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.397+0000 7fe97c7aa700 1 Processor -- start 2022-09-20T20:07:10.399 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.397+0000 7fe97c7aa700 1 -- start start 2022-09-20T20:07:10.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.398+0000 7fe97c7aa700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe97406bad0 0x7fe9740cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:10.403 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.398+0000 7fe97c7aa700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe9740acdd0 0x7fe9740c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:10.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.398+0000 7fe97c7aa700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97415f850 con 0x7fe97406bea0 2022-09-20T20:07:10.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.398+0000 7fe97c7aa700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9740cbe50 con 0x7fe9740acdd0 2022-09-20T20:07:10.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.398+0000 7fe97c7aa700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9740ce7e0 con 0x7fe97406bad0 2022-09-20T20:07:10.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.398+0000 7fe979d45700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe9740acdd0 0x7fe9740c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:10.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe979d45700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe9740acdd0 0x7fe9740c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.36:51796/0 (socket says 172.21.15.36:51796) 2022-09-20T20:07:10.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe97ad47700 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe97406bea0 0x7fe97415a9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.36:48256/0 (socket says 172.21.15.36:48256) 2022-09-20T20:07:10.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe979d45700 1 -- 172.21.15.36:0/621144029 learned_addr learned my addr 172.21.15.36:0/621144029 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:10.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe97a546700 1 --2- 172.21.15.36:0/621144029 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe97406bad0 0x7fe9740cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:10.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe979d45700 1 -- 172.21.15.36:0/621144029 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe97406bad0 msgr2=0x7fe9740cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:10.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe979d45700 1 --2- 172.21.15.36:0/621144029 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe97406bad0 0x7fe9740cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:10.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe979d45700 1 -- 172.21.15.36:0/621144029 >> v1:172.21.15.115:6789/0 conn(0x7fe97406bea0 legacy=0x7fe97415a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:10.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe979d45700 1 -- 172.21.15.36:0/621144029 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97415faf0 con 0x7fe9740acdd0 2022-09-20T20:07:10.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe979d45700 1 --2- 172.21.15.36:0/621144029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9740acdd0 0x7fe9740c93a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe9700098f0 tx=0x7fe970009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8262b32d323a234b server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:10.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe979544700 1 -- 172.21.15.36:0/621144029 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970011830 con 0x7fe9740acdd0 2022-09-20T20:07:10.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe979544700 1 -- 172.21.15.36:0/621144029 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe970011990 con 0x7fe9740acdd0 2022-09-20T20:07:10.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.399+0000 7fe979544700 1 -- 172.21.15.36:0/621144029 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970011d20 con 0x7fe9740acdd0 2022-09-20T20:07:10.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.400+0000 7fe97c7aa700 1 -- 172.21.15.36:0/621144029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9740acdd0 msgr2=0x7fe9740c93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:10.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.400+0000 7fe97c7aa700 1 --2- 172.21.15.36:0/621144029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9740acdd0 0x7fe9740c93a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe9700098f0 tx=0x7fe970009c00 comp rx=0 tx=0).stop 2022-09-20T20:07:10.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.400+0000 7fe97c7aa700 1 -- 172.21.15.36:0/621144029 shutdown_connections 2022-09-20T20:07:10.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.400+0000 7fe97c7aa700 1 --2- 172.21.15.36:0/621144029 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe97406bad0 0x7fe9740cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:10.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.400+0000 7fe97c7aa700 1 --2- 172.21.15.36:0/621144029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9740acdd0 0x7fe9740c93a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:10.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.400+0000 7fe97c7aa700 1 -- 172.21.15.36:0/621144029 >> 172.21.15.36:0/621144029 conn(0x7fe9740cb440 msgr2=0x7fe9740cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:10.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.400+0000 7fe97c7aa700 1 -- 172.21.15.36:0/621144029 shutdown_connections 2022-09-20T20:07:10.410 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.400+0000 7fe97c7aa700 1 -- 172.21.15.36:0/621144029 wait complete. 2022-09-20T20:07:10.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.401+0000 7fe97c7aa700 1 Processor -- start 2022-09-20T20:07:10.411 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.401+0000 7fe97c7aa700 1 -- start start 2022-09-20T20:07:10.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97c7aa700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe97406bad0 0x7fe9740b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:10.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97c7aa700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe97406bea0 0x7fe9740c2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:10.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97c7aa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9740acdd0 0x7fe9740c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:10.419 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97c7aa700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9741f2ad0 con 0x7fe97406bad0 2022-09-20T20:07:10.420 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97c7aa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9740b10b0 con 0x7fe9740acdd0 2022-09-20T20:07:10.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97c7aa700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9741f1ec0 con 0x7fe97406bea0 2022-09-20T20:07:10.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97a546700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe97406bad0 0x7fe9740b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:10.421 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97a546700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe97406bad0 0x7fe9740b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58184/0 (socket says 172.21.15.36:58184) 2022-09-20T20:07:10.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97a546700 1 -- 172.21.15.36:0/2875903389 learned_addr learned my addr 172.21.15.36:0/2875903389 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:10.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97ad47700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9740acdd0 0x7fe9740c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:10.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe979d45700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe97406bea0 0x7fe9740c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:10.422 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97a546700 1 -- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe97406bea0 msgr2=0x7fe9740c2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:10.423 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97a546700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe97406bea0 0x7fe9740c2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:10.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97a546700 1 -- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9740acdd0 msgr2=0x7fe9740c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:10.424 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97a546700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9740acdd0 0x7fe9740c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:10.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97a546700 1 -- 172.21.15.36:0/2875903389 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe970009550 con 0x7fe97406bad0 2022-09-20T20:07:10.425 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.402+0000 7fe97a546700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe97406bad0 0x7fe9740b0b70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e870 tx=0x7fe96400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:10.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.403+0000 7fe962ffd700 1 -- 172.21.15.36:0/2875903389 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964009590 con 0x7fe97406bad0 2022-09-20T20:07:10.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.403+0000 7fe962ffd700 1 -- 172.21.15.36:0/2875903389 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96400ee70 con 0x7fe97406bad0 2022-09-20T20:07:10.426 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.403+0000 7fe97c7aa700 1 -- 172.21.15.36:0/2875903389 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9741f21a0 con 0x7fe97406bad0 2022-09-20T20:07:10.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.403+0000 7fe962ffd700 1 -- 172.21.15.36:0/2875903389 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96400ca50 con 0x7fe97406bad0 2022-09-20T20:07:10.427 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.403+0000 7fe97c7aa700 1 -- 172.21.15.36:0/2875903389 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9741f2760 con 0x7fe97406bad0 2022-09-20T20:07:10.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.404+0000 7fe97c7aa700 1 -- 172.21.15.36:0/2875903389 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9740663c0 con 0x7fe97406bad0 2022-09-20T20:07:10.428 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.404+0000 7fe962ffd700 1 -- 172.21.15.36:0/2875903389 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7fe964019030 con 0x7fe97406bad0 2022-09-20T20:07:10.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.404+0000 7fe962ffd700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fe954065a50 0x7fe954067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:10.429 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.404+0000 7fe962ffd700 1 -- 172.21.15.36:0/2875903389 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fe9640824d0 con 0x7fe97406bad0 2022-09-20T20:07:10.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.405+0000 7fe979d45700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fe954065a50 0x7fe954067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:10.430 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.405+0000 7fe979d45700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fe954065a50 0x7fe954067f00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe9700098f0 tx=0x7fe970008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:10.431 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.409+0000 7fe962ffd700 1 -- 172.21.15.36:0/2875903389 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7fe9640506a0 con 0x7fe97406bad0 2022-09-20T20:07:10.603 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:10.601+0000 7fe97c7aa700 1 -- 172.21.15.36:0/2875903389 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9740ceac0 con 0x7fe97406bad0 2022-09-20T20:07:11.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.255+0000 7fe962ffd700 1 -- 172.21.15.36:0/2875903389 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7fe964053cc0 con 0x7fe97406bad0 2022-09-20T20:07:11.261 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 -- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fe954065a50 msgr2=0x7fe954067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fe954065a50 0x7fe954067f00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe9700098f0 tx=0x7fe970008030 comp rx=0 tx=0).stop 2022-09-20T20:07:11.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 -- 172.21.15.36:0/2875903389 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe97406bad0 msgr2=0x7fe9740b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.262 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe97406bad0 0x7fe9740b0b70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e870 tx=0x7fe96400ec30 comp rx=0 tx=0).stop 2022-09-20T20:07:11.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 -- 172.21.15.36:0/2875903389 shutdown_connections 2022-09-20T20:07:11.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7fe954065a50 0x7fe954067f00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.263 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9740acdd0 0x7fe9740c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7fe97406bad0 0x7fe9740b0b70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 --2- 172.21.15.36:0/2875903389 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7fe97406bea0 0x7fe9740c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 -- 172.21.15.36:0/2875903389 >> 172.21.15.36:0/2875903389 conn(0x7fe9740cb440 msgr2=0x7fe97415cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:11.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 -- 172.21.15.36:0/2875903389 shutdown_connections 2022-09-20T20:07:11.264 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.259+0000 7fe97c7aa700 1 -- 172.21.15.36:0/2875903389 wait complete. 2022-09-20T20:07:11.265 INFO:teuthology.orchestra.run.smithi036.stderr:enabled application 'rbd' on pool 'rbd' 2022-09-20T20:07:11.278 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-09-20T20:07:11.279 INFO:tasks.ceph.mds.a:Restarting daemon 2022-09-20T20:07:11.279 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-09-20T20:07:11.282 INFO:tasks.ceph.mds.a:Started 2022-09-20T20:07:11.283 INFO:tasks.ceph.mds.c:Restarting daemon 2022-09-20T20:07:11.283 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-09-20T20:07:11.285 INFO:tasks.ceph.mds.c:Started 2022-09-20T20:07:11.286 INFO:tasks.ceph.mds.b:Restarting daemon 2022-09-20T20:07:11.286 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-09-20T20:07:11.292 INFO:tasks.ceph.mds.b:Started 2022-09-20T20:07:11.293 INFO:tasks.ceph.mds.d:Restarting daemon 2022-09-20T20:07:11.293 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2022-09-20T20:07:11.296 INFO:tasks.ceph.mds.d:Started 2022-09-20T20:07:11.297 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-09-20T20:07:11.297 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-09-20T20:07:11.372 INFO:tasks.ceph.mds.a.smithi036.stdout:starting mds.a at 2022-09-20T20:07:11.374 INFO:tasks.ceph.mds.c.smithi036.stdout:starting mds.c at 2022-09-20T20:07:11.376 INFO:tasks.ceph.mds.d.smithi115.stdout:starting mds.d at 2022-09-20T20:07:11.377 INFO:tasks.ceph.mds.b.smithi115.stdout:starting mds.b at 2022-09-20T20:07:11.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.437+0000 7f5d0143a700 1 Processor -- start 2022-09-20T20:07:11.439 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.437+0000 7f5d0143a700 1 -- start start 2022-09-20T20:07:11.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.438+0000 7f5d0143a700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5cfc0c8580 0x7f5cfc0ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.442 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.438+0000 7f5d0143a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5cfc0ac030 0x7f5cfc15e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.438+0000 7f5d0143a700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cfc069e70 con 0x7f5cfc0c8950 2022-09-20T20:07:11.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.438+0000 7f5d0143a700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc15fa90 con 0x7f5cfc0ac030 2022-09-20T20:07:11.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.438+0000 7f5d0143a700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0aa8f0 con 0x7f5cfc0c8580 2022-09-20T20:07:11.443 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.438+0000 7f5cfb7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5cfc0ac030 0x7f5cfc15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.438+0000 7f5cfb7fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f5cfc0ac030 0x7f5cfc15e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58222/0 (socket says 172.21.15.36:58222) 2022-09-20T20:07:11.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.438+0000 7f5cfb7fe700 1 -- 172.21.15.36:0/1850443953 learned_addr learned my addr 172.21.15.36:0/1850443953 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:11.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.438+0000 7f5cfaffd700 1 --2- 172.21.15.36:0/1850443953 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5cfc0c8580 0x7f5cfc0ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.444 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5cfb7fe700 1 -- 172.21.15.36:0/1850443953 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5cfc0c8580 msgr2=0x7f5cfc0ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:11.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5cfb7fe700 1 --2- 172.21.15.36:0/1850443953 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5cfc0c8580 0x7f5cfc0ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5cfb7fe700 1 -- 172.21.15.36:0/1850443953 >> v1:172.21.15.115:6789/0 conn(0x7f5cfc0c8950 legacy=0x7f5cfc0696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5cf9ffb700 1 -- 172.21.15.36:0/1850443953 <== mon.1 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088770345 0 0) 0x7f5cfc069e70 con 0x7f5cfc0c8950 2022-09-20T20:07:11.445 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5cfb7fe700 1 -- 172.21.15.36:0/1850443953 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc06a150 con 0x7f5cfc0ac030 2022-09-20T20:07:11.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5cfb7fe700 1 --2- 172.21.15.36:0/1850443953 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 0x7f5cfc15e540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098f0 tx=0x7f5cec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95c95fb7586df1e server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:11.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5cf9ffb700 1 -- 172.21.15.36:0/1850443953 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec011830 con 0x7f5cfc0ac030 2022-09-20T20:07:11.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5cf9ffb700 1 -- 172.21.15.36:0/1850443953 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec011990 con 0x7f5cfc0ac030 2022-09-20T20:07:11.446 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5cf9ffb700 1 -- 172.21.15.36:0/1850443953 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec011d20 con 0x7f5cfc0ac030 2022-09-20T20:07:11.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5d0143a700 1 -- 172.21.15.36:0/1850443953 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 msgr2=0x7f5cfc15e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5d0143a700 1 --2- 172.21.15.36:0/1850443953 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 0x7f5cfc15e540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098f0 tx=0x7f5cec009c00 comp rx=0 tx=0).stop 2022-09-20T20:07:11.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5d0143a700 1 -- 172.21.15.36:0/1850443953 shutdown_connections 2022-09-20T20:07:11.447 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5d0143a700 1 --2- 172.21.15.36:0/1850443953 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 0x7f5cfc15e540 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5d0143a700 1 --2- 172.21.15.36:0/1850443953 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5cfc0c8580 0x7f5cfc0ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5d0143a700 1 -- 172.21.15.36:0/1850443953 >> 172.21.15.36:0/1850443953 conn(0x7f5cfc0ce950 msgr2=0x7f5cfc0ca740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:11.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.439+0000 7f5d0143a700 1 -- 172.21.15.36:0/1850443953 shutdown_connections 2022-09-20T20:07:11.448 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.440+0000 7f5d0143a700 1 -- 172.21.15.36:0/1850443953 wait complete. 2022-09-20T20:07:11.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.441+0000 7f5d0143a700 1 Processor -- start 2022-09-20T20:07:11.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.441+0000 7f5d0143a700 1 -- start start 2022-09-20T20:07:11.452 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.442+0000 7f5d0143a700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 0x7f5cfc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.442+0000 7f5d0143a700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5cfc0c8580 0x7f5cfc157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.442+0000 7f5d0143a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc0c8950 0x7f5cfc0c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.442+0000 7f5d0143a700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc06d990 con 0x7f5cfc0ac030 2022-09-20T20:07:11.453 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.442+0000 7f5d0143a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0c5170 con 0x7f5cfc0c8950 2022-09-20T20:07:11.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.442+0000 7f5d0143a700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0c5450 con 0x7f5cfc0c8580 2022-09-20T20:07:11.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.442+0000 7f5cfaffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 0x7f5cfc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.442+0000 7f5cfaffd700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 0x7f5cfc06d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58224/0 (socket says 172.21.15.36:58224) 2022-09-20T20:07:11.454 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.442+0000 7f5cfaffd700 1 -- 172.21.15.36:0/2241113840 learned_addr learned my addr 172.21.15.36:0/2241113840 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:11.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.442+0000 7f5cfb7fe700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc0c8950 0x7f5cfc0c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5cfa7fc700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5cfc0c8580 0x7f5cfc157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5cfaffd700 1 -- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5cfc0c8580 msgr2=0x7f5cfc157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.455 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5cfaffd700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5cfc0c8580 0x7f5cfc157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5cfaffd700 1 -- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc0c8950 msgr2=0x7f5cfc0c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5cfaffd700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc0c8950 0x7f5cfc0c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5cfaffd700 1 -- 172.21.15.36:0/2241113840 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec009550 con 0x7f5cfc0ac030 2022-09-20T20:07:11.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5cfb7fe700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc0c8950 0x7f5cfc0c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-09-20T20:07:11.456 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5cfaffd700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 0x7f5cfc06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400efc0 tx=0x7f5cf400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:11.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5ce37fe700 1 -- 172.21.15.36:0/2241113840 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4009ba0 con 0x7f5cfc0ac030 2022-09-20T20:07:11.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5ce37fe700 1 -- 172.21.15.36:0/2241113840 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf400f040 con 0x7f5cfc0ac030 2022-09-20T20:07:11.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5ce37fe700 1 -- 172.21.15.36:0/2241113840 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4014480 con 0x7f5cfc0ac030 2022-09-20T20:07:11.457 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5d0143a700 1 -- 172.21.15.36:0/2241113840 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc0c5730 con 0x7f5cfc0ac030 2022-09-20T20:07:11.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.443+0000 7f5d0143a700 1 -- 172.21.15.36:0/2241113840 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc0c5cf0 con 0x7f5cfc0ac030 2022-09-20T20:07:11.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.444+0000 7f5d0143a700 1 -- 172.21.15.36:0/2241113840 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc0c9d80 con 0x7f5cfc0ac030 2022-09-20T20:07:11.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.444+0000 7f5ce37fe700 1 -- 172.21.15.36:0/2241113840 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7f5cf40145e0 con 0x7f5cfc0ac030 2022-09-20T20:07:11.458 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.445+0000 7f5ce37fe700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f5cdc065a50 0x7f5cdc067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.445+0000 7f5ce37fe700 1 -- 172.21.15.36:0/2241113840 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5cf4083110 con 0x7f5cfc0ac030 2022-09-20T20:07:11.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.445+0000 7f5cfa7fc700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f5cdc065a50 0x7f5cdc067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.445+0000 7f5cfa7fc700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f5cdc065a50 0x7f5cdc067f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0cdae0 tx=0x7f5cf0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:11.459 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.449+0000 7f5ce37fe700 1 -- 172.21.15.36:0/2241113840 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f5cf40512d0 con 0x7f5cfc0ac030 2022-09-20T20:07:11.645 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.644+0000 7f5d0143a700 1 -- 172.21.15.36:0/2241113840 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cfc0715a0 con 0x7f5cfc0ac030 2022-09-20T20:07:11.646 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.645+0000 7f5ce37fe700 1 -- 172.21.15.36:0/2241113840 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13708 (secure 0 0 0) 0x7f5cf40548f0 con 0x7f5cfc0ac030 2022-09-20T20:07:11.647 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:07:11.647 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":18,"fsid":"3c118203-7ef2-4e20-8eae-6178b988d685","created":"2022-09-20T20:06:51.484163+0000","modified":"2022-09-20T20:07:11.252635+0000","last_up_change":"2022-09-20T20:07:02.800559+0000","last_in_change":"2022-09-20T20:06:54.021671+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-09-20T20:07:02.886289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-09-20T20:07:09.970087+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"027a07c2-8eec-415a-9e2b-ae85e68a47ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":99855},{"type":"v1","addr":"172.21.15.36:6809","nonce":99855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":99855},{"type":"v1","addr":"172.21.15.36:6811","nonce":99855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":99855},{"type":"v1","addr":"172.21.15.36:6815","nonce":99855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":99855},{"type":"v1","addr":"172.21.15.36:6813","nonce":99855}]},"public_addr":"172.21.15.36:6809/99855","cluster_addr":"172.21.15.36:6811/99855","heartbeat_back_addr":"172.21.15.36:6815/99855","heartbeat_front_addr":"172.21.15.36:6813/99855","state":["exists","up"]},{"osd":1,"uuid":"73600a5c-a12c-460c-80ec-6fcd76cc6f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6800","nonce":99858},{"type":"v1","addr":"172.21.15.36:6801","nonce":99858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":99858},{"type":"v1","addr":"172.21.15.36:6803","nonce":99858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":99858},{"type":"v1","addr":"172.21.15.36:6807","nonce":99858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":99858},{"type":"v1","addr":"172.21.15.36:6805","nonce":99858}]},"public_addr":"172.21.15.36:6801/99858","cluster_addr":"172.21.15.36:6803/99858","heartbeat_back_addr":"172.21.15.36:6807/99858","heartbeat_front_addr":"172.21.15.36:6805/99858","state":["exists","up"]},{"osd":2,"uuid":"76ffb9bc-7fa7-4cdd-8190-8caebf90ccc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":99871},{"type":"v1","addr":"172.21.15.36:6817","nonce":99871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":99871},{"type":"v1","addr":"172.21.15.36:6819","nonce":99871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":99871},{"type":"v1","addr":"172.21.15.36:6823","nonce":99871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":99871},{"type":"v1","addr":"172.21.15.36:6821","nonce":99871}]},"public_addr":"172.21.15.36:6817/99871","cluster_addr":"172.21.15.36:6819/99871","heartbeat_back_addr":"172.21.15.36:6823/99871","heartbeat_front_addr":"172.21.15.36:6821/99871","state":["exists","up"]},{"osd":3,"uuid":"dd9de39a-4958-4162-9a85-4d950e22837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":99872},{"type":"v1","addr":"172.21.15.36:6825","nonce":99872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":99872},{"type":"v1","addr":"172.21.15.36:6827","nonce":99872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":99872},{"type":"v1","addr":"172.21.15.36:6831","nonce":99872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":99872},{"type":"v1","addr":"172.21.15.36:6829","nonce":99872}]},"public_addr":"172.21.15.36:6825/99872","cluster_addr":"172.21.15.36:6827/99872","heartbeat_back_addr":"172.21.15.36:6831/99872","heartbeat_front_addr":"172.21.15.36:6829/99872","state":["exists","up"]},{"osd":4,"uuid":"8a19aa0f-2885-42c5-a36f-a1dc44bd5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":100090},{"type":"v1","addr":"172.21.15.115:6801","nonce":100090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":100090},{"type":"v1","addr":"172.21.15.115:6805","nonce":100090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":100090},{"type":"v1","addr":"172.21.15.115:6813","nonce":100090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":100090},{"type":"v1","addr":"172.21.15.115:6809","nonce":100090}]},"public_addr":"172.21.15.115:6801/100090","cluster_addr":"172.21.15.115:6805/100090","heartbeat_back_addr":"172.21.15.115:6813/100090","heartbeat_front_addr":"172.21.15.115:6809/100090","state":["exists","up"]},{"osd":5,"uuid":"e12282e4-18b8-4bd3-887c-bd913650b1c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":100091},{"type":"v1","addr":"172.21.15.115:6817","nonce":100091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":100091},{"type":"v1","addr":"172.21.15.115:6819","nonce":100091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":100091},{"type":"v1","addr":"172.21.15.115:6823","nonce":100091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":100091},{"type":"v1","addr":"172.21.15.115:6821","nonce":100091}]},"public_addr":"172.21.15.115:6817/100091","cluster_addr":"172.21.15.115:6819/100091","heartbeat_back_addr":"172.21.15.115:6823/100091","heartbeat_front_addr":"172.21.15.115:6821/100091","state":["exists","up"]},{"osd":6,"uuid":"23dd01f7-1bd4-43de-9231-bd81bf351537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":100092},{"type":"v1","addr":"172.21.15.115:6804","nonce":100092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":100092},{"type":"v1","addr":"172.21.15.115:6808","nonce":100092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":100092},{"type":"v1","addr":"172.21.15.115:6815","nonce":100092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":100092},{"type":"v1","addr":"172.21.15.115:6812","nonce":100092}]},"public_addr":"172.21.15.115:6804/100092","cluster_addr":"172.21.15.115:6808/100092","heartbeat_back_addr":"172.21.15.115:6815/100092","heartbeat_front_addr":"172.21.15.115:6812/100092","state":["exists","up"]},{"osd":7,"uuid":"0414b75c-e580-4955-90ab-adec5f32acbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":100135},{"type":"v1","addr":"172.21.15.115:6825","nonce":100135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":100135},{"type":"v1","addr":"172.21.15.115:6827","nonce":100135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":100135},{"type":"v1","addr":"172.21.15.115:6831","nonce":100135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":100135},{"type":"v1","addr":"172.21.15.115:6829","nonce":100135}]},"public_addr":"172.21.15.115:6825/100135","cluster_addr":"172.21.15.115:6827/100135","heartbeat_back_addr":"172.21.15.115:6831/100135","heartbeat_front_addr":"172.21.15.115:6829/100135","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.122892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:58.979821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.136461+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.220479+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:58.971449+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.275615+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.200553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.332596+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-20T20:07:11.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 -- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f5cdc065a50 msgr2=0x7f5cdc067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f5cdc065a50 0x7f5cdc067f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0cdae0 tx=0x7f5cf0009040 comp rx=0 tx=0).stop 2022-09-20T20:07:11.649 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 -- 172.21.15.36:0/2241113840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 msgr2=0x7f5cfc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 0x7f5cfc06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400efc0 tx=0x7f5cf400c3a0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 -- 172.21.15.36:0/2241113840 shutdown_connections 2022-09-20T20:07:11.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f5cdc065a50 0x7f5cdc067f00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.650 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cfc0c8950 0x7f5cfc0c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f5cfc0ac030 0x7f5cfc06d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 --2- 172.21.15.36:0/2241113840 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f5cfc0c8580 0x7f5cfc157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 -- 172.21.15.36:0/2241113840 >> 172.21.15.36:0/2241113840 conn(0x7f5cfc0ce950 msgr2=0x7f5cfc15f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:11.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 -- 172.21.15.36:0/2241113840 shutdown_connections 2022-09-20T20:07:11.651 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.647+0000 7f5d0143a700 1 -- 172.21.15.36:0/2241113840 wait complete. 2022-09-20T20:07:11.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-09-20T20:07:02.886289+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-09-20T20:07:09.970087+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-09-20T20:07:11.662 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-09-20T20:07:11.789 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.787+0000 7f55efdb4700 1 Processor -- start 2022-09-20T20:07:11.790 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.787+0000 7f55efdb4700 1 -- start start 2022-09-20T20:07:11.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.788+0000 7f55efdb4700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55e806fa50 0x7f55e806cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.803 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.788+0000 7f55efdb4700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e815f460 con 0x7f55e806fe20 2022-09-20T20:07:11.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.788+0000 7f55efdb4700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e815ee50 con 0x7f55e815f8f0 2022-09-20T20:07:11.804 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.788+0000 7f55efdb4700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e815fd50 con 0x7f55e806fa50 2022-09-20T20:07:11.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.788+0000 7f55ed34f700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f55e806fe20 0x7f55e80ca130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:37670/0 (socket says 172.21.15.36:37670) 2022-09-20T20:07:11.805 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.788+0000 7f55ed34f700 1 -- 172.21.15.36:0/207627036 learned_addr learned my addr 172.21.15.36:0/207627036 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-09-20T20:07:11.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55edb50700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55e806fa50 0x7f55e806cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.806 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874034799 0 0) 0x7f55e815f460 con 0x7f55e806fe20 2022-09-20T20:07:11.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55d00035b0 con 0x7f55e806fe20 2022-09-20T20:07:11.807 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 <== mon.1 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534339960 0 0) 0x7f55e815ee50 con 0x7f55e815f8f0 2022-09-20T20:07:11.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55e815f460 con 0x7f55e815f8f0 2022-09-20T20:07:11.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1507911115 0 0) 0x7f55d00035b0 con 0x7f55e806fe20 2022-09-20T20:07:11.808 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55e806fa50 msgr2=0x7f55e806cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:11.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55e806fa50 0x7f55e806cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.809 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 >> v1:172.21.15.115:6789/0 conn(0x7f55e815f8f0 legacy=0x7f55e815ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e806d360 con 0x7f55e806fe20 2022-09-20T20:07:11.810 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3393640461 0 0) 0x7f55e4001670 con 0x7f55e806fe20 2022-09-20T20:07:11.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 >> v1:172.21.15.36:6789/0 conn(0x7f55e806fe20 legacy=0x7f55e80ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.811 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55d0003bc0 0x7f55d0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55e806fa50 0x7f55d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55d0004d90 0x7f55d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.812 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d000bac0 con 0x7f55d0004d90 2022-09-20T20:07:11.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d000bc00 con 0x7f55d0003bc0 2022-09-20T20:07:11.813 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d000bdc0 con 0x7f55e806fa50 2022-09-20T20:07:11.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ee351700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55d0004d90 0x7f55d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.814 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55edb50700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55d0003bc0 0x7f55d0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ed34f700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55e806fa50 0x7f55d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.815 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ee351700 1 -- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55e806fa50 msgr2=0x7f55d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:11.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ee351700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55e806fa50 0x7f55d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ee351700 1 -- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55d0003bc0 msgr2=0x7f55d0003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:11.816 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ee351700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55d0003bc0 0x7f55d0003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.789+0000 7f55ee351700 1 -- 172.21.15.36:0/207627036 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d00035b0 con 0x7f55d0004d90 2022-09-20T20:07:11.817 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55ee351700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55d0004d90 0x7f55d0005200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f55e0002230 tx=0x7f55d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422efa8c2bcb5af4 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:11.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0001f30 con 0x7f55d0004d90 2022-09-20T20:07:11.818 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e000ce80 con 0x7f55d0004d90 2022-09-20T20:07:11.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55ecb4e700 1 -- 172.21.15.36:0/207627036 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0003c40 con 0x7f55d0004d90 2022-09-20T20:07:11.819 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55efdb4700 1 -- 172.21.15.36:0/207627036 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55d0004d90 msgr2=0x7f55d0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55efdb4700 1 --2- 172.21.15.36:0/207627036 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55d0004d90 0x7f55d0005200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f55e0002230 tx=0x7f55d0005880 comp rx=0 tx=0).stop 2022-09-20T20:07:11.820 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55ee351700 1 -- 172.21.15.36:0/207627036 reap_dead start 2022-09-20T20:07:11.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55efdb4700 1 -- 172.21.15.36:0/207627036 shutdown_connections 2022-09-20T20:07:11.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55efdb4700 1 -- 172.21.15.36:0/207627036 >> 172.21.15.36:0/207627036 conn(0x7f55e80cde30 msgr2=0x7f55e80ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:11.821 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55efdb4700 1 -- 172.21.15.36:0/207627036 shutdown_connections 2022-09-20T20:07:11.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.790+0000 7f55efdb4700 1 -- 172.21.15.36:0/207627036 wait complete. 2022-09-20T20:07:11.822 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.792+0000 7f55efdb4700 1 Processor -- start 2022-09-20T20:07:11.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.792+0000 7f55efdb4700 1 -- start start 2022-09-20T20:07:11.823 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55efdb4700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55d0004d90 0x7f55e814bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55efdb4700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55e80c3020 0x7f55e80c3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55efdb4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e81580e0 0x7f55e8158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.824 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55efdb4700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8158ac0 con 0x7f55e80c3020 2022-09-20T20:07:11.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55efdb4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80c39d0 con 0x7f55e81580e0 2022-09-20T20:07:11.825 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55efdb4700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e814c170 con 0x7f55d0004d90 2022-09-20T20:07:11.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55ed34f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55e80c3020 0x7f55e80c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.826 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55ed34f700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55e80c3020 0x7f55e80c3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58246/0 (socket says 172.21.15.36:58246) 2022-09-20T20:07:11.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55ed34f700 1 -- 172.21.15.36:0/3458115636 learned_addr learned my addr 172.21.15.36:0/3458115636 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:11.827 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55ee351700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e81580e0 0x7f55e8158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55edb50700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55d0004d90 0x7f55e814bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55ed34f700 1 -- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55d0004d90 msgr2=0x7f55e814bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.828 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55ed34f700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55d0004d90 0x7f55e814bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55ed34f700 1 -- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e81580e0 msgr2=0x7f55e8158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.829 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55ed34f700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e81580e0 0x7f55e8158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55ed34f700 1 -- 172.21.15.36:0/3458115636 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e80c4c80 con 0x7f55e80c3020 2022-09-20T20:07:11.830 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.793+0000 7f55edb50700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55d0004d90 0x7f55e814bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-09-20T20:07:11.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.794+0000 7f55ed34f700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55e80c3020 0x7f55e80c3490 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f55e4007290 tx=0x7f55e40075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:11.831 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.794+0000 7f55d67fc700 1 -- 172.21.15.36:0/3458115636 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4011580 con 0x7f55e80c3020 2022-09-20T20:07:11.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.794+0000 7f55d67fc700 1 -- 172.21.15.36:0/3458115636 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e4011720 con 0x7f55e80c3020 2022-09-20T20:07:11.832 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.794+0000 7f55efdb4700 1 -- 172.21.15.36:0/3458115636 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e80c3be0 con 0x7f55e80c3020 2022-09-20T20:07:11.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.794+0000 7f55d67fc700 1 -- 172.21.15.36:0/3458115636 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4011ad0 con 0x7f55e80c3020 2022-09-20T20:07:11.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.794+0000 7f55efdb4700 1 -- 172.21.15.36:0/3458115636 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e80c41a0 con 0x7f55e80c3020 2022-09-20T20:07:11.833 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.795+0000 7f55efdb4700 1 -- 172.21.15.36:0/3458115636 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e80663c0 con 0x7f55e80c3020 2022-09-20T20:07:11.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.795+0000 7f55d67fc700 1 -- 172.21.15.36:0/3458115636 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7f55e4011c30 con 0x7f55e80c3020 2022-09-20T20:07:11.834 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.795+0000 7f55d67fc700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f55c8065a50 0x7f55c8067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:11.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.796+0000 7f55d67fc700 1 -- 172.21.15.36:0/3458115636 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f55e4082710 con 0x7f55e80c3020 2022-09-20T20:07:11.835 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.796+0000 7f55edb50700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f55c8065a50 0x7f55c8067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:11.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.796+0000 7f55edb50700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f55c8065a50 0x7f55c8067f00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009c60 tx=0x7f55d8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:11.836 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.800+0000 7f55d67fc700 1 -- 172.21.15.36:0/3458115636 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f55e4050980 con 0x7f55e80c3020 2022-09-20T20:07:11.992 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.990+0000 7f55efdb4700 1 -- 172.21.15.36:0/3458115636 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55e815f0f0 con 0x7f55e80c3020 2022-09-20T20:07:11.993 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.990+0000 7f55d67fc700 1 -- 172.21.15.36:0/3458115636 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55e4053fa0 con 0x7f55e80c3020 2022-09-20T20:07:11.993 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-09-20T20:07:11.995 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 -- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f55c8065a50 msgr2=0x7f55c8067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f55c8065a50 0x7f55c8067f00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009c60 tx=0x7f55d8008040 comp rx=0 tx=0).stop 2022-09-20T20:07:11.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 -- 172.21.15.36:0/3458115636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55e80c3020 msgr2=0x7f55e80c3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:11.996 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55e80c3020 0x7f55e80c3490 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f55e4007290 tx=0x7f55e40075a0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 -- 172.21.15.36:0/3458115636 shutdown_connections 2022-09-20T20:07:11.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f55c8065a50 0x7f55c8067f00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e81580e0 0x7f55e8158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.997 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f55d0004d90 0x7f55e814bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 --2- 172.21.15.36:0/3458115636 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f55e80c3020 0x7f55e80c3490 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:11.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 -- 172.21.15.36:0/3458115636 >> 172.21.15.36:0/3458115636 conn(0x7f55e80cde30 msgr2=0x7f55e80cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:11.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 -- 172.21.15.36:0/3458115636 shutdown_connections 2022-09-20T20:07:11.998 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:11.992+0000 7f55efdb4700 1 -- 172.21.15.36:0/3458115636 wait complete. 2022-09-20T20:07:12.006 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-09-20T20:07:12.132 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.130+0000 7f371ca98700 1 Processor -- start 2022-09-20T20:07:12.133 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.131+0000 7f371ca98700 1 -- start start 2022-09-20T20:07:12.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.131+0000 7f371ca98700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ab2f0 0x7f37180cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.135 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.131+0000 7f371ca98700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f37180ac6d0 0x7f37180aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.131+0000 7f371ca98700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37180adaf0 con 0x7f37180ad5a0 2022-09-20T20:07:12.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.131+0000 7f371ca98700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37180aced0 con 0x7f37180ac6d0 2022-09-20T20:07:12.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.131+0000 7f371ca98700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37180ad1b0 con 0x7f37180ab2f0 2022-09-20T20:07:12.136 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.132+0000 7f3716d9d700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f37180ac6d0 0x7f37180aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.132+0000 7f371659c700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ab2f0 0x7f37180cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.132+0000 7f3715d9b700 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f37180ad5a0 0x7f37180ae540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.36:48326/0 (socket says 172.21.15.36:48326) 2022-09-20T20:07:12.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.132+0000 7f3715d9b700 1 -- 172.21.15.36:0/421971703 learned_addr learned my addr 172.21.15.36:0/421971703 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-09-20T20:07:12.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.132+0000 7f3716d9d700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f37180ac6d0 0x7f37180aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.36:51870/0 (socket says 172.21.15.36:51870) 2022-09-20T20:07:12.137 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.132+0000 7f3716d9d700 1 -- 172.21.15.36:0/421971703 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ab2f0 msgr2=0x7f37180cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:12.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.132+0000 7f3716d9d700 1 --2- 172.21.15.36:0/421971703 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ab2f0 0x7f37180cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.132+0000 7f3716d9d700 1 -- 172.21.15.36:0/421971703 >> v1:172.21.15.115:6789/0 conn(0x7f37180ad5a0 legacy=0x7f37180ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.138 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.132+0000 7f3716d9d700 1 -- 172.21.15.36:0/421971703 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37180add40 con 0x7f37180ac6d0 2022-09-20T20:07:12.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.132+0000 7f3716d9d700 1 --2- 172.21.15.36:0/421971703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37180ac6d0 0x7f37180aec50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3710009d90 tx=0x7f371000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489a59b27885962b server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:12.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371559a700 1 -- 172.21.15.36:0/421971703 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371000e070 con 0x7f37180ac6d0 2022-09-20T20:07:12.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371559a700 1 -- 172.21.15.36:0/421971703 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371000bb20 con 0x7f37180ac6d0 2022-09-20T20:07:12.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371559a700 1 -- 172.21.15.36:0/421971703 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710012430 con 0x7f37180ac6d0 2022-09-20T20:07:12.139 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371ca98700 1 -- 172.21.15.36:0/421971703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37180ac6d0 msgr2=0x7f37180aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371ca98700 1 --2- 172.21.15.36:0/421971703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37180ac6d0 0x7f37180aec50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3710009d90 tx=0x7f371000b330 comp rx=0 tx=0).stop 2022-09-20T20:07:12.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371ca98700 1 -- 172.21.15.36:0/421971703 shutdown_connections 2022-09-20T20:07:12.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371ca98700 1 --2- 172.21.15.36:0/421971703 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ab2f0 0x7f37180cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371ca98700 1 --2- 172.21.15.36:0/421971703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37180ac6d0 0x7f37180aec50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.140 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371ca98700 1 -- 172.21.15.36:0/421971703 >> 172.21.15.36:0/421971703 conn(0x7f371806a6d0 msgr2=0x7f37180ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:12.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371ca98700 1 -- 172.21.15.36:0/421971703 shutdown_connections 2022-09-20T20:07:12.141 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.133+0000 7f371ca98700 1 -- 172.21.15.36:0/421971703 wait complete. 2022-09-20T20:07:12.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.134+0000 7f371ca98700 1 Processor -- start 2022-09-20T20:07:12.144 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.135+0000 7f371ca98700 1 -- start start 2022-09-20T20:07:12.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.135+0000 7f371ca98700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37180ab2f0 0x7f3718155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.135+0000 7f371ca98700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ac6d0 0x7f37181e2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.135+0000 7f371ca98700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37180ad5a0 0x7f37180c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.145 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.135+0000 7f371ca98700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37181e31b0 con 0x7f37180ad5a0 2022-09-20T20:07:12.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.135+0000 7f371ca98700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37181562d0 con 0x7f37180ab2f0 2022-09-20T20:07:12.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.135+0000 7f371ca98700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37181e2ed0 con 0x7f37180ac6d0 2022-09-20T20:07:12.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.135+0000 7f3716d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37180ad5a0 0x7f37180c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.135+0000 7f3716d9d700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37180ad5a0 0x7f37180c22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58260/0 (socket says 172.21.15.36:58260) 2022-09-20T20:07:12.146 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.135+0000 7f3716d9d700 1 -- 172.21.15.36:0/196046413 learned_addr learned my addr 172.21.15.36:0/196046413 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:12.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f371659c700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37180ab2f0 0x7f3718155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3715d9b700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ac6d0 0x7f37181e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3716d9d700 1 -- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ac6d0 msgr2=0x7f37181e2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.147 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3716d9d700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ac6d0 0x7f37181e2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3716d9d700 1 -- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37180ab2f0 msgr2=0x7f3718155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3716d9d700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37180ab2f0 0x7f3718155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3716d9d700 1 -- 172.21.15.36:0/196046413 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3710009a70 con 0x7f37180ad5a0 2022-09-20T20:07:12.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f371659c700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37180ab2f0 0x7f3718155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-09-20T20:07:12.148 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3715d9b700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ac6d0 0x7f37181e2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-09-20T20:07:12.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3716d9d700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37180ad5a0 0x7f37180c22b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f371000b370 tx=0x7f3710004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:12.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3706ffd700 1 -- 172.21.15.36:0/196046413 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710012430 con 0x7f37180ad5a0 2022-09-20T20:07:12.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3706ffd700 1 -- 172.21.15.36:0/196046413 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3710020070 con 0x7f37180ad5a0 2022-09-20T20:07:12.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f371ca98700 1 -- 172.21.15.36:0/196046413 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3718159170 con 0x7f37180ad5a0 2022-09-20T20:07:12.149 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f3706ffd700 1 -- 172.21.15.36:0/196046413 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710004280 con 0x7f37180ad5a0 2022-09-20T20:07:12.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.136+0000 7f371ca98700 1 -- 172.21.15.36:0/196046413 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3718159730 con 0x7f37180ad5a0 2022-09-20T20:07:12.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.137+0000 7f371ca98700 1 -- 172.21.15.36:0/196046413 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3718158fd0 con 0x7f37180ad5a0 2022-09-20T20:07:12.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.137+0000 7f3706ffd700 1 -- 172.21.15.36:0/196046413 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7f37100043e0 con 0x7f37180ad5a0 2022-09-20T20:07:12.150 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.138+0000 7f3706ffd700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f36f4065a50 0x7f36f4067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.138+0000 7f3706ffd700 1 -- 172.21.15.36:0/196046413 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3710022070 con 0x7f37180ad5a0 2022-09-20T20:07:12.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.138+0000 7f371659c700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f36f4065a50 0x7f36f4067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.139+0000 7f371659c700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f36f4065a50 0x7f36f4067f00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f370c009ac0 tx=0x7f370c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:12.151 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.142+0000 7f3706ffd700 1 -- 172.21.15.36:0/196046413 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f3718158fd0 con 0x7f37180ad5a0 2022-09-20T20:07:12.331 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.329+0000 7f371ca98700 1 -- 172.21.15.36:0/196046413 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3718158fd0 con 0x7f37180ad5a0 2022-09-20T20:07:12.332 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.329+0000 7f3706ffd700 1 -- 172.21.15.36:0/196046413 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3718158fd0 con 0x7f37180ad5a0 2022-09-20T20:07:12.333 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-09-20T20:07:12.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 -- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f36f4065a50 msgr2=0x7f36f4067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.335 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f36f4065a50 0x7f36f4067f00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f370c009ac0 tx=0x7f370c008040 comp rx=0 tx=0).stop 2022-09-20T20:07:12.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 -- 172.21.15.36:0/196046413 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37180ad5a0 msgr2=0x7f37180c22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.336 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37180ad5a0 0x7f37180c22b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f371000b370 tx=0x7f3710004ad0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 -- 172.21.15.36:0/196046413 shutdown_connections 2022-09-20T20:07:12.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f36f4065a50 0x7f36f4067f00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.337 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f37180ad5a0 0x7f37180c22b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37180ab2f0 0x7f3718155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 --2- 172.21.15.36:0/196046413 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f37180ac6d0 0x7f37181e2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.338 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 -- 172.21.15.36:0/196046413 >> 172.21.15.36:0/196046413 conn(0x7f371806a6d0 msgr2=0x7f37180cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:12.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 -- 172.21.15.36:0/196046413 shutdown_connections 2022-09-20T20:07:12.339 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.331+0000 7f371ca98700 1 -- 172.21.15.36:0/196046413 wait complete. 2022-09-20T20:07:12.345 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-09-20T20:07:12.471 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.469+0000 7f9547cfe700 1 Processor -- start 2022-09-20T20:07:12.472 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.470+0000 7f9547cfe700 1 -- start start 2022-09-20T20:07:12.477 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.470+0000 7f9547cfe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9540157da0 0x7f954015c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.470+0000 7f9547cfe700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f95400cea20 0x7f95400c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.470+0000 7f9547cfe700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954015c780 con 0x7f95400cac30 2022-09-20T20:07:12.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.470+0000 7f9547cfe700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9540159180 con 0x7f9540157da0 2022-09-20T20:07:12.478 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.470+0000 7f9547cfe700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954006ff50 con 0x7f95400cea20 2022-09-20T20:07:12.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9545a9a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9540157da0 0x7f954015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9545a9a700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f9540157da0 0x7f954015c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58268/0 (socket says 172.21.15.36:58268) 2022-09-20T20:07:12.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9545a9a700 1 -- 172.21.15.36:0/157313237 learned_addr learned my addr 172.21.15.36:0/157313237 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:12.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f954629b700 1 --2- 172.21.15.36:0/157313237 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f95400cea20 0x7f95400c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.479 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9545a9a700 1 -- 172.21.15.36:0/157313237 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f95400cea20 msgr2=0x7f95400c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:12.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9545a9a700 1 --2- 172.21.15.36:0/157313237 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f95400cea20 0x7f95400c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9545a9a700 1 -- 172.21.15.36:0/157313237 >> v1:172.21.15.115:6789/0 conn(0x7f95400cac30 legacy=0x7f95400ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:12.480 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9545a9a700 1 -- 172.21.15.36:0/157313237 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95400cb160 con 0x7f9540157da0 2022-09-20T20:07:12.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9545a9a700 1 --2- 172.21.15.36:0/157313237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 0x7f954015c240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f953c00bb90 tx=0x7f953c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7835bc54415898 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:12.481 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9544a98700 1 -- 172.21.15.36:0/157313237 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00dd50 con 0x7f9540157da0 2022-09-20T20:07:12.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9544a98700 1 -- 172.21.15.36:0/157313237 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c010470 con 0x7f9540157da0 2022-09-20T20:07:12.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.471+0000 7f9544a98700 1 -- 172.21.15.36:0/157313237 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c010800 con 0x7f9540157da0 2022-09-20T20:07:12.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.472+0000 7f9547cfe700 1 -- 172.21.15.36:0/157313237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 msgr2=0x7f954015c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.482 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.472+0000 7f9547cfe700 1 --2- 172.21.15.36:0/157313237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 0x7f954015c240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f953c00bb90 tx=0x7f953c00bea0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.472+0000 7f9547cfe700 1 -- 172.21.15.36:0/157313237 shutdown_connections 2022-09-20T20:07:12.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.472+0000 7f9547cfe700 1 --2- 172.21.15.36:0/157313237 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f95400cea20 0x7f95400c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.472+0000 7f9547cfe700 1 --2- 172.21.15.36:0/157313237 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 0x7f954015c240 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.472+0000 7f9547cfe700 1 -- 172.21.15.36:0/157313237 >> 172.21.15.36:0/157313237 conn(0x7f954006c020 msgr2=0x7f954006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:12.483 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.472+0000 7f9547cfe700 1 -- 172.21.15.36:0/157313237 shutdown_connections 2022-09-20T20:07:12.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.472+0000 7f9547cfe700 1 -- 172.21.15.36:0/157313237 wait complete. 2022-09-20T20:07:12.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.473+0000 7f9547cfe700 1 Processor -- start 2022-09-20T20:07:12.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.473+0000 7f9547cfe700 1 -- start start 2022-09-20T20:07:12.484 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f9547cfe700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f95400cac30 0x7f9540152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f9547cfe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95400cea20 0x7f95400c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f9547cfe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 0x7f9540151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.485 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f9547cfe700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95400c3280 con 0x7f9540157da0 2022-09-20T20:07:12.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f9547cfe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9540152380 con 0x7f95400cea20 2022-09-20T20:07:12.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f9547cfe700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95400c4680 con 0x7f95400cac30 2022-09-20T20:07:12.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f954629b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 0x7f9540151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f954629b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 0x7f9540151e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58278/0 (socket says 172.21.15.36:58278) 2022-09-20T20:07:12.486 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f954629b700 1 -- 172.21.15.36:0/2984309370 learned_addr learned my addr 172.21.15.36:0/2984309370 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:12.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f9545299700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95400cea20 0x7f95400c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f9545a9a700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f95400cac30 0x7f9540152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.487 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f954629b700 1 -- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f95400cac30 msgr2=0x7f9540152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f954629b700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f95400cac30 0x7f9540152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f954629b700 1 -- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95400cea20 msgr2=0x7f95400c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f954629b700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95400cea20 0x7f95400c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f954629b700 1 -- 172.21.15.36:0/2984309370 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c00b7f0 con 0x7f9540157da0 2022-09-20T20:07:12.488 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f954629b700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 0x7f9540151e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9538007e40 tx=0x7f953800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:12.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f952e7fc700 1 -- 172.21.15.36:0/2984309370 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953800ed30 con 0x7f9540157da0 2022-09-20T20:07:12.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.474+0000 7f952e7fc700 1 -- 172.21.15.36:0/2984309370 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9538014420 con 0x7f9540157da0 2022-09-20T20:07:12.489 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.475+0000 7f9547cfe700 1 -- 172.21.15.36:0/2984309370 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95400c43a0 con 0x7f9540157da0 2022-09-20T20:07:12.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.475+0000 7f952e7fc700 1 -- 172.21.15.36:0/2984309370 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95380147d0 con 0x7f9540157da0 2022-09-20T20:07:12.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.475+0000 7f9547cfe700 1 -- 172.21.15.36:0/2984309370 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95400c2ea0 con 0x7f9540157da0 2022-09-20T20:07:12.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.475+0000 7f9547cfe700 1 -- 172.21.15.36:0/2984309370 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95400c33c0 con 0x7f9540157da0 2022-09-20T20:07:12.490 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.476+0000 7f952e7fc700 1 -- 172.21.15.36:0/2984309370 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7f9538012050 con 0x7f9540157da0 2022-09-20T20:07:12.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.476+0000 7f952e7fc700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f9520065a50 0x7f9520067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.476+0000 7f952e7fc700 1 -- 172.21.15.36:0/2984309370 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9538082c80 con 0x7f9540157da0 2022-09-20T20:07:12.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.476+0000 7f9545a9a700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f9520065a50 0x7f9520067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.492 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.477+0000 7f9545a9a700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f9520065a50 0x7f9520067f00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f953c00bee0 tx=0x7f953c013000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:12.493 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.480+0000 7f952e7fc700 1 -- 172.21.15.36:0/2984309370 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f95400c33c0 con 0x7f9540157da0 2022-09-20T20:07:12.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.713+0000 7f9547cfe700 1 -- 172.21.15.36:0/2984309370 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95400c33c0 con 0x7f9540157da0 2022-09-20T20:07:12.715 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.713+0000 7f952e7fc700 1 -- 172.21.15.36:0/2984309370 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f95400c33c0 con 0x7f9540157da0 2022-09-20T20:07:12.716 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:07:12.716 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6833/3140741233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6832","nonce":3140741233},{"type":"v1","addr":"172.21.15.36:6833","nonce":3140741233}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.36:6835/112811358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6834","nonce":112811358},{"type":"v1","addr":"172.21.15.36:6835","nonce":112811358}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4321,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1687185302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1687185302},{"type":"v1","addr":"172.21.15.115:6836","nonce":1687185302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4324,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/371771105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":371771105},{"type":"v1","addr":"172.21.15.115:6837","nonce":371771105}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2022-09-20T20:07:12.717 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 -- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f9520065a50 msgr2=0x7f9520067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f9520065a50 0x7f9520067f00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f953c00bee0 tx=0x7f953c013000 comp rx=0 tx=0).stop 2022-09-20T20:07:12.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 -- 172.21.15.36:0/2984309370 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 msgr2=0x7f9540151e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 0x7f9540151e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9538007e40 tx=0x7f953800e3c0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.718 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 -- 172.21.15.36:0/2984309370 shutdown_connections 2022-09-20T20:07:12.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f9520065a50 0x7f9520067f00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f9540157da0 0x7f9540151e40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.719 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f95400cac30 0x7f9540152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 --2- 172.21.15.36:0/2984309370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95400cea20 0x7f95400c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 -- 172.21.15.36:0/2984309370 >> 172.21.15.36:0/2984309370 conn(0x7f954006c020 msgr2=0x7f95400ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:12.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 -- 172.21.15.36:0/2984309370 shutdown_connections 2022-09-20T20:07:12.720 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.715+0000 7f9547cfe700 1 -- 172.21.15.36:0/2984309370 wait complete. 2022-09-20T20:07:12.720 INFO:teuthology.orchestra.run.smithi036.stderr:dumped fsmap epoch 3 2022-09-20T20:07:12.729 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-09-20T20:07:12.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.853+0000 7f05fddb0700 1 Processor -- start 2022-09-20T20:07:12.855 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.853+0000 7f05fddb0700 1 -- start start 2022-09-20T20:07:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05fddb0700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f05f80c8580 0x7f05f80ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.858 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05fddb0700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 0x7f05f80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05fddb0700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f8069bf0 con 0x7f05f80ab020 2022-09-20T20:07:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05fddb0700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f80aa8f0 con 0x7f05f80c8580 2022-09-20T20:07:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05fddb0700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8069e70 con 0x7f05f80c8950 2022-09-20T20:07:12.859 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05f77fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f05f80c8580 0x7f05f80ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05f7fff700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f05f80ab020 0x7f05f80ab4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:37724/0 (socket says 172.21.15.36:37724) 2022-09-20T20:07:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05f7fff700 1 -- 172.21.15.36:0/3834319286 learned_addr learned my addr 172.21.15.36:0/3834319286 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-09-20T20:07:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05f6ffd700 1 --2- 172.21.15.36:0/3834319286 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 0x7f05f80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.860 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05f77fe700 1 -- 172.21.15.36:0/3834319286 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 msgr2=0x7f05f80696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05f77fe700 1 --2- 172.21.15.36:0/3834319286 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 0x7f05f80696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05f77fe700 1 -- 172.21.15.36:0/3834319286 >> v1:172.21.15.36:6789/0 conn(0x7f05f80ab020 legacy=0x7f05f80ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05f77fe700 1 -- 172.21.15.36:0/3834319286 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f806a150 con 0x7f05f80c8580 2022-09-20T20:07:12.861 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05f67fc700 1 -- 172.21.15.36:0/3834319286 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447204785 0 0) 0x7f05f8069bf0 con 0x7f05f80ab020 2022-09-20T20:07:12.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.854+0000 7f05f77fe700 1 --2- 172.21.15.36:0/3834319286 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80c8580 0x7f05f80ad050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009840 tx=0x7f05e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31601b56f3c62218 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:12.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05f67fc700 1 -- 172.21.15.36:0/3834319286 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0011830 con 0x7f05f80c8580 2022-09-20T20:07:12.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05f67fc700 1 -- 172.21.15.36:0/3834319286 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e0011990 con 0x7f05f80c8580 2022-09-20T20:07:12.862 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05f67fc700 1 -- 172.21.15.36:0/3834319286 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0011d20 con 0x7f05f80c8580 2022-09-20T20:07:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05fddb0700 1 -- 172.21.15.36:0/3834319286 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80c8580 msgr2=0x7f05f80ad050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05fddb0700 1 --2- 172.21.15.36:0/3834319286 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80c8580 0x7f05f80ad050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009840 tx=0x7f05e0009b50 comp rx=0 tx=0).stop 2022-09-20T20:07:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05fddb0700 1 -- 172.21.15.36:0/3834319286 shutdown_connections 2022-09-20T20:07:12.863 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05fddb0700 1 --2- 172.21.15.36:0/3834319286 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80c8580 0x7f05f80ad050 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05fddb0700 1 --2- 172.21.15.36:0/3834319286 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 0x7f05f80696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05fddb0700 1 -- 172.21.15.36:0/3834319286 >> 172.21.15.36:0/3834319286 conn(0x7f05f80ce950 msgr2=0x7f05f80ca740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:12.864 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05fddb0700 1 -- 172.21.15.36:0/3834319286 shutdown_connections 2022-09-20T20:07:12.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.855+0000 7f05fddb0700 1 -- 172.21.15.36:0/3834319286 wait complete. 2022-09-20T20:07:12.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.856+0000 7f05fddb0700 1 Processor -- start 2022-09-20T20:07:12.865 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05fddb0700 1 -- start start 2022-09-20T20:07:12.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05fddb0700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80ab020 0x7f05f80c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05fddb0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f80c8580 0x7f05f814b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05fddb0700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 0x7f05f81f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.869 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05fddb0700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f806b7a0 con 0x7f05f80ab020 2022-09-20T20:07:12.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05fddb0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f81f5000 con 0x7f05f80c8580 2022-09-20T20:07:12.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05fddb0700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80c4700 con 0x7f05f80c8950 2022-09-20T20:07:12.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05f77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80ab020 0x7f05f80c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.870 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05f77fe700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80ab020 0x7f05f80c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58288/0 (socket says 172.21.15.36:58288) 2022-09-20T20:07:12.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05f77fe700 1 -- 172.21.15.36:0/445436435 learned_addr learned my addr 172.21.15.36:0/445436435 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:12.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05f6ffd700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f80c8580 0x7f05f814b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.857+0000 7f05f7fff700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 0x7f05f81f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05f77fe700 1 -- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 msgr2=0x7f05f81f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.871 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05f77fe700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 0x7f05f81f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05f77fe700 1 -- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f80c8580 msgr2=0x7f05f814b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:12.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05f77fe700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f80c8580 0x7f05f814b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:12.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05f77fe700 1 -- 172.21.15.36:0/445436435 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00094a0 con 0x7f05f80ab020 2022-09-20T20:07:12.872 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05f7fff700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 0x7f05f81f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-09-20T20:07:12.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05f77fe700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80ab020 0x7f05f80c41c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f05e00090b0 tx=0x7f05e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:12.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05effff700 1 -- 172.21.15.36:0/445436435 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e00038a0 con 0x7f05f80ab020 2022-09-20T20:07:12.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05effff700 1 -- 172.21.15.36:0/445436435 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e001e070 con 0x7f05f80ab020 2022-09-20T20:07:12.873 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05fddb0700 1 -- 172.21.15.36:0/445436435 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f80b33b0 con 0x7f05f80ab020 2022-09-20T20:07:12.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05effff700 1 -- 172.21.15.36:0/445436435 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0003c20 con 0x7f05f80ab020 2022-09-20T20:07:12.874 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.858+0000 7f05fddb0700 1 -- 172.21.15.36:0/445436435 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f80b3970 con 0x7f05f80ab020 2022-09-20T20:07:12.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.859+0000 7f05fddb0700 1 -- 172.21.15.36:0/445436435 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f80c9d80 con 0x7f05f80ab020 2022-09-20T20:07:12.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.859+0000 7f05effff700 1 -- 172.21.15.36:0/445436435 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7f05e000f050 con 0x7f05f80ab020 2022-09-20T20:07:12.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.860+0000 7f05effff700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f05dc065a50 0x7f05dc067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:12.875 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.860+0000 7f05effff700 1 -- 172.21.15.36:0/445436435 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f05e0083830 con 0x7f05f80ab020 2022-09-20T20:07:12.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.860+0000 7f05f6ffd700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f05dc065a50 0x7f05dc067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:12.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.860+0000 7f05f6ffd700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f05dc065a50 0x7f05dc067f00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f05f8159830 tx=0x7f05e8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:12.876 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:12.864+0000 7f05effff700 1 -- 172.21.15.36:0/445436435 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f05e00519f0 con 0x7f05f80ab020 2022-09-20T20:07:13.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.057+0000 7f05fddb0700 1 -- 172.21.15.36:0/445436435 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05f80b3b60 con 0x7f05f80ab020 2022-09-20T20:07:13.059 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.058+0000 7f05effff700 1 -- 172.21.15.36:0/445436435 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13695 (secure 0 0 0) 0x7f05e0020020 con 0x7f05f80ab020 2022-09-20T20:07:13.060 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:07:13.060 INFO:teuthology.orchestra.run.smithi036.stdout:{"epoch":19,"fsid":"3c118203-7ef2-4e20-8eae-6178b988d685","created":"2022-09-20T20:06:51.484163+0000","modified":"2022-09-20T20:07:12.254144+0000","last_up_change":"2022-09-20T20:07:02.800559+0000","last_in_change":"2022-09-20T20:06:54.021671+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-09-20T20:07:02.886289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-09-20T20:07:09.970087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"027a07c2-8eec-415a-9e2b-ae85e68a47ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6808","nonce":99855},{"type":"v1","addr":"172.21.15.36:6809","nonce":99855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6810","nonce":99855},{"type":"v1","addr":"172.21.15.36:6811","nonce":99855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6814","nonce":99855},{"type":"v1","addr":"172.21.15.36:6815","nonce":99855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6812","nonce":99855},{"type":"v1","addr":"172.21.15.36:6813","nonce":99855}]},"public_addr":"172.21.15.36:6809/99855","cluster_addr":"172.21.15.36:6811/99855","heartbeat_back_addr":"172.21.15.36:6815/99855","heartbeat_front_addr":"172.21.15.36:6813/99855","state":["exists","up"]},{"osd":1,"uuid":"73600a5c-a12c-460c-80ec-6fcd76cc6f3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6800","nonce":99858},{"type":"v1","addr":"172.21.15.36:6801","nonce":99858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6802","nonce":99858},{"type":"v1","addr":"172.21.15.36:6803","nonce":99858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6806","nonce":99858},{"type":"v1","addr":"172.21.15.36:6807","nonce":99858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6804","nonce":99858},{"type":"v1","addr":"172.21.15.36:6805","nonce":99858}]},"public_addr":"172.21.15.36:6801/99858","cluster_addr":"172.21.15.36:6803/99858","heartbeat_back_addr":"172.21.15.36:6807/99858","heartbeat_front_addr":"172.21.15.36:6805/99858","state":["exists","up"]},{"osd":2,"uuid":"76ffb9bc-7fa7-4cdd-8190-8caebf90ccc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6816","nonce":99871},{"type":"v1","addr":"172.21.15.36:6817","nonce":99871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6818","nonce":99871},{"type":"v1","addr":"172.21.15.36:6819","nonce":99871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6822","nonce":99871},{"type":"v1","addr":"172.21.15.36:6823","nonce":99871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6820","nonce":99871},{"type":"v1","addr":"172.21.15.36:6821","nonce":99871}]},"public_addr":"172.21.15.36:6817/99871","cluster_addr":"172.21.15.36:6819/99871","heartbeat_back_addr":"172.21.15.36:6823/99871","heartbeat_front_addr":"172.21.15.36:6821/99871","state":["exists","up"]},{"osd":3,"uuid":"dd9de39a-4958-4162-9a85-4d950e22837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6824","nonce":99872},{"type":"v1","addr":"172.21.15.36:6825","nonce":99872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6826","nonce":99872},{"type":"v1","addr":"172.21.15.36:6827","nonce":99872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6830","nonce":99872},{"type":"v1","addr":"172.21.15.36:6831","nonce":99872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.36:6828","nonce":99872},{"type":"v1","addr":"172.21.15.36:6829","nonce":99872}]},"public_addr":"172.21.15.36:6825/99872","cluster_addr":"172.21.15.36:6827/99872","heartbeat_back_addr":"172.21.15.36:6831/99872","heartbeat_front_addr":"172.21.15.36:6829/99872","state":["exists","up"]},{"osd":4,"uuid":"8a19aa0f-2885-42c5-a36f-a1dc44bd5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":100090},{"type":"v1","addr":"172.21.15.115:6801","nonce":100090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6803","nonce":100090},{"type":"v1","addr":"172.21.15.115:6805","nonce":100090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":100090},{"type":"v1","addr":"172.21.15.115:6813","nonce":100090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6807","nonce":100090},{"type":"v1","addr":"172.21.15.115:6809","nonce":100090}]},"public_addr":"172.21.15.115:6801/100090","cluster_addr":"172.21.15.115:6805/100090","heartbeat_back_addr":"172.21.15.115:6813/100090","heartbeat_front_addr":"172.21.15.115:6809/100090","state":["exists","up"]},{"osd":5,"uuid":"e12282e4-18b8-4bd3-887c-bd913650b1c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":100091},{"type":"v1","addr":"172.21.15.115:6817","nonce":100091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":100091},{"type":"v1","addr":"172.21.15.115:6819","nonce":100091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":100091},{"type":"v1","addr":"172.21.15.115:6823","nonce":100091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":100091},{"type":"v1","addr":"172.21.15.115:6821","nonce":100091}]},"public_addr":"172.21.15.115:6817/100091","cluster_addr":"172.21.15.115:6819/100091","heartbeat_back_addr":"172.21.15.115:6823/100091","heartbeat_front_addr":"172.21.15.115:6821/100091","state":["exists","up"]},{"osd":6,"uuid":"23dd01f7-1bd4-43de-9231-bd81bf351537","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":100092},{"type":"v1","addr":"172.21.15.115:6804","nonce":100092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":100092},{"type":"v1","addr":"172.21.15.115:6808","nonce":100092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":100092},{"type":"v1","addr":"172.21.15.115:6815","nonce":100092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":100092},{"type":"v1","addr":"172.21.15.115:6812","nonce":100092}]},"public_addr":"172.21.15.115:6804/100092","cluster_addr":"172.21.15.115:6808/100092","heartbeat_back_addr":"172.21.15.115:6815/100092","heartbeat_front_addr":"172.21.15.115:6812/100092","state":["exists","up"]},{"osd":7,"uuid":"0414b75c-e580-4955-90ab-adec5f32acbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":100135},{"type":"v1","addr":"172.21.15.115:6825","nonce":100135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":100135},{"type":"v1","addr":"172.21.15.115:6827","nonce":100135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":100135},{"type":"v1","addr":"172.21.15.115:6831","nonce":100135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":100135},{"type":"v1","addr":"172.21.15.115:6829","nonce":100135}]},"public_addr":"172.21.15.115:6825/100135","cluster_addr":"172.21.15.115:6827/100135","heartbeat_back_addr":"172.21.15.115:6831/100135","heartbeat_front_addr":"172.21.15.115:6829/100135","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.122892+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:58.979821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.136461+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.220479+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:58.971449+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.275615+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.200553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-20T20:06:59.332596+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-20T20:07:13.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.059+0000 7f05fddb0700 1 -- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f05dc065a50 msgr2=0x7f05dc067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.059+0000 7f05fddb0700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f05dc065a50 0x7f05dc067f00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f05f8159830 tx=0x7f05e8008040 comp rx=0 tx=0).stop 2022-09-20T20:07:13.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.059+0000 7f05fddb0700 1 -- 172.21.15.36:0/445436435 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80ab020 msgr2=0x7f05f80c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.062 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.059+0000 7f05fddb0700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80ab020 0x7f05f80c41c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f05e00090b0 tx=0x7f05e0004750 comp rx=0 tx=0).stop 2022-09-20T20:07:13.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.059+0000 7f05fddb0700 1 -- 172.21.15.36:0/445436435 shutdown_connections 2022-09-20T20:07:13.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.059+0000 7f05fddb0700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f05dc065a50 0x7f05dc067f00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.059+0000 7f05fddb0700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f05f80c8950 0x7f05f81f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.059+0000 7f05fddb0700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f05f80ab020 0x7f05f80c41c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.063 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.059+0000 7f05fddb0700 1 --2- 172.21.15.36:0/445436435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05f80c8580 0x7f05f814b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.060+0000 7f05fddb0700 1 -- 172.21.15.36:0/445436435 >> 172.21.15.36:0/445436435 conn(0x7f05f80ce950 msgr2=0x7f05f8069060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:13.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.060+0000 7f05fddb0700 1 -- 172.21.15.36:0/445436435 shutdown_connections 2022-09-20T20:07:13.064 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.060+0000 7f05fddb0700 1 -- 172.21.15.36:0/445436435 wait complete. 2022-09-20T20:07:13.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-09-20T20:07:02.886289+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-09-20T20:07:09.970087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-09-20T20:07:13.075 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-09-20T20:07:13.203 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.201+0000 7f7b84d60700 1 Processor -- start 2022-09-20T20:07:13.204 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.201+0000 7f7b84d60700 1 -- start start 2022-09-20T20:07:13.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b84d60700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7b8006fa50 0x7f7b8006cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.209 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b84d60700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8015f8f0 0x7f7b8015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b84d60700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8015fd00 con 0x7f7b8006fe20 2022-09-20T20:07:13.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b84d60700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8015f460 con 0x7f7b8006fa50 2022-09-20T20:07:13.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b84d60700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8015ee50 con 0x7f7b8015f8f0 2022-09-20T20:07:13.210 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b7e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7b8006fa50 0x7f7b8006cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b7e59c700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f7b8006fa50 0x7f7b8006cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58296/0 (socket says 172.21.15.36:58296) 2022-09-20T20:07:13.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b7e59c700 1 -- 172.21.15.36:0/1385145599 learned_addr learned my addr 172.21.15.36:0/1385145599 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:13.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b7ed9d700 1 --2- 172.21.15.36:0/1385145599 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8015f8f0 0x7f7b8015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.211 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b7e59c700 1 -- 172.21.15.36:0/1385145599 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8015f8f0 msgr2=0x7f7b8015ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:13.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b7e59c700 1 --2- 172.21.15.36:0/1385145599 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8015f8f0 0x7f7b8015ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b7e59c700 1 -- 172.21.15.36:0/1385145599 >> v1:172.21.15.115:6789/0 conn(0x7f7b8006fe20 legacy=0x7f7b800ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b7e59c700 1 -- 172.21.15.36:0/1385145599 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8006d360 con 0x7f7b8006fa50 2022-09-20T20:07:13.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b7ed9d700 1 --2- 172.21.15.36:0/1385145599 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8015f8f0 0x7f7b8015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-09-20T20:07:13.212 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.202+0000 7f7b7e59c700 1 --2- 172.21.15.36:0/1385145599 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fa50 0x7f7b8006cd80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7b700098f0 tx=0x7f7b70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c3e2106750e3f8 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:13.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b7d59a700 1 -- 172.21.15.36:0/1385145599 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000b8a0 con 0x7f7b8006fa50 2022-09-20T20:07:13.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b7d59a700 1 -- 172.21.15.36:0/1385145599 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7000ba00 con 0x7f7b8006fa50 2022-09-20T20:07:13.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b7d59a700 1 -- 172.21.15.36:0/1385145599 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000bd90 con 0x7f7b8006fa50 2022-09-20T20:07:13.213 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b84d60700 1 -- 172.21.15.36:0/1385145599 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fa50 msgr2=0x7f7b8006cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b84d60700 1 --2- 172.21.15.36:0/1385145599 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fa50 0x7f7b8006cd80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7b700098f0 tx=0x7f7b70009c00 comp rx=0 tx=0).stop 2022-09-20T20:07:13.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b84d60700 1 -- 172.21.15.36:0/1385145599 shutdown_connections 2022-09-20T20:07:13.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b84d60700 1 --2- 172.21.15.36:0/1385145599 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8015f8f0 0x7f7b8015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.214 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b84d60700 1 --2- 172.21.15.36:0/1385145599 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fa50 0x7f7b8006cd80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b84d60700 1 -- 172.21.15.36:0/1385145599 >> 172.21.15.36:0/1385145599 conn(0x7f7b800cde30 msgr2=0x7f7b800ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:13.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b84d60700 1 -- 172.21.15.36:0/1385145599 shutdown_connections 2022-09-20T20:07:13.215 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.203+0000 7f7b84d60700 1 -- 172.21.15.36:0/1385145599 wait complete. 2022-09-20T20:07:13.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b84d60700 1 Processor -- start 2022-09-20T20:07:13.216 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b84d60700 1 -- start start 2022-09-20T20:07:13.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b84d60700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8006fa50 0x7f7b801f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b84d60700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fe20 0x7f7b8014eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b84d60700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b8015f8f0 0x7f7b801f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.217 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b84d60700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801f1050 con 0x7f7b8006fe20 2022-09-20T20:07:13.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b84d60700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801f19b0 con 0x7f7b8015f8f0 2022-09-20T20:07:13.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b84d60700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801f09a0 con 0x7f7b8006fa50 2022-09-20T20:07:13.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b7dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fe20 0x7f7b8014eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.218 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b7dd9b700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fe20 0x7f7b8014eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58304/0 (socket says 172.21.15.36:58304) 2022-09-20T20:07:13.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b7dd9b700 1 -- 172.21.15.36:0/2283578433 learned_addr learned my addr 172.21.15.36:0/2283578433 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:13.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.205+0000 7f7b7e59c700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8006fa50 0x7f7b801f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b7ed9d700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b8015f8f0 0x7f7b801f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.219 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b7dd9b700 1 -- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8006fa50 msgr2=0x7f7b801f1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b7dd9b700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8006fa50 0x7f7b801f1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b7dd9b700 1 -- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b8015f8f0 msgr2=0x7f7b801f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b7dd9b700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b8015f8f0 0x7f7b801f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b7dd9b700 1 -- 172.21.15.36:0/2283578433 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70009550 con 0x7f7b8006fe20 2022-09-20T20:07:13.220 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b7dd9b700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fe20 0x7f7b8014eab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400c4e0 tx=0x7f7b7400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:13.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b6effd700 1 -- 172.21.15.36:0/2283578433 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740052d0 con 0x7f7b8006fe20 2022-09-20T20:07:13.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b6effd700 1 -- 172.21.15.36:0/2283578433 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b74005430 con 0x7f7b8006fe20 2022-09-20T20:07:13.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b84d60700 1 -- 172.21.15.36:0/2283578433 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b801f49f0 con 0x7f7b8006fe20 2022-09-20T20:07:13.221 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b6effd700 1 -- 172.21.15.36:0/2283578433 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740038c0 con 0x7f7b8006fe20 2022-09-20T20:07:13.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.206+0000 7f7b84d60700 1 -- 172.21.15.36:0/2283578433 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b801f0c70 con 0x7f7b8006fe20 2022-09-20T20:07:13.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.207+0000 7f7b84d60700 1 -- 172.21.15.36:0/2283578433 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b800663c0 con 0x7f7b8006fe20 2022-09-20T20:07:13.222 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.208+0000 7f7b6effd700 1 -- 172.21.15.36:0/2283578433 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7f7b74019030 con 0x7f7b8006fe20 2022-09-20T20:07:13.223 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.208+0000 7f7b6effd700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f7b5c065a50 0x7f7b5c067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.208+0000 7f7b6effd700 1 -- 172.21.15.36:0/2283578433 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7b74081b30 con 0x7f7b8006fe20 2022-09-20T20:07:13.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.209+0000 7f7b7e59c700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f7b5c065a50 0x7f7b5c067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.209+0000 7f7b7e59c700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f7b5c065a50 0x7f7b5c067f00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b70014040 tx=0x7f7b70008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:13.224 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.212+0000 7f7b6effd700 1 -- 172.21.15.36:0/2283578433 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f7b7404fd90 con 0x7f7b8006fe20 2022-09-20T20:07:13.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.400+0000 7f7b84d60700 1 -- 172.21.15.36:0/2283578433 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b8015fd00 con 0x7f7b8006fe20 2022-09-20T20:07:13.402 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.400+0000 7f7b6effd700 1 -- 172.21.15.36:0/2283578433 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b8015fd00 con 0x7f7b8006fe20 2022-09-20T20:07:13.403 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 1 2022-09-20T20:07:13.404 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 -- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f7b5c065a50 msgr2=0x7f7b5c067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f7b5c065a50 0x7f7b5c067f00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7b70014040 tx=0x7f7b70008030 comp rx=0 tx=0).stop 2022-09-20T20:07:13.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 -- 172.21.15.36:0/2283578433 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fe20 msgr2=0x7f7b8014eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fe20 0x7f7b8014eab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400c4e0 tx=0x7f7b7400c7f0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.405 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 -- 172.21.15.36:0/2283578433 shutdown_connections 2022-09-20T20:07:13.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f7b5c065a50 0x7f7b5c067f00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b8015f8f0 0x7f7b801f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f7b8006fa50 0x7f7b801f1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.406 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 --2- 172.21.15.36:0/2283578433 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f7b8006fe20 0x7f7b8014eab0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 -- 172.21.15.36:0/2283578433 >> 172.21.15.36:0/2283578433 conn(0x7f7b800cde30 msgr2=0x7f7b800c8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:13.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 -- 172.21.15.36:0/2283578433 shutdown_connections 2022-09-20T20:07:13.407 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.402+0000 7f7b84d60700 1 -- 172.21.15.36:0/2283578433 wait complete. 2022-09-20T20:07:13.417 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-09-20T20:07:13.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.543+0000 7f2237855700 1 Processor -- start 2022-09-20T20:07:13.545 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.543+0000 7f2237855700 1 -- start start 2022-09-20T20:07:13.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f2237855700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2230159250 0x7f22300cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.548 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f2237855700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f22300cba10 0x7f22300cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f2237855700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22300cf630 con 0x7f22300c9d90 2022-09-20T20:07:13.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f2237855700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22300ca970 con 0x7f2230159250 2022-09-20T20:07:13.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f2237855700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300cac50 con 0x7f22300cba10 2022-09-20T20:07:13.549 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f22355f1700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f2230159250 0x7f22300cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f2235df2700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f22300c9d90 0x7f22300cc060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:37754/0 (socket says 172.21.15.36:37754) 2022-09-20T20:07:13.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f2235df2700 1 -- 172.21.15.36:0/2870998159 learned_addr learned my addr 172.21.15.36:0/2870998159 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-09-20T20:07:13.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f2234df0700 1 --2- 172.21.15.36:0/2870998159 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f22300cba10 0x7f22300cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.550 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f22355f1700 1 -- 172.21.15.36:0/2870998159 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f22300cba10 msgr2=0x7f22300cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:13.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f22355f1700 1 --2- 172.21.15.36:0/2870998159 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f22300cba10 0x7f22300cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f22355f1700 1 -- 172.21.15.36:0/2870998159 >> v1:172.21.15.36:6789/0 conn(0x7f22300c9d90 legacy=0x7f22300cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f22355f1700 1 -- 172.21.15.36:0/2870998159 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300caf30 con 0x7f2230159250 2022-09-20T20:07:13.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.544+0000 7f221ffff700 1 -- 172.21.15.36:0/2870998159 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039027544 0 0) 0x7f22300cf630 con 0x7f22300c9d90 2022-09-20T20:07:13.551 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f22355f1700 1 --2- 172.21.15.36:0/2870998159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2230159250 0x7f22300cd310 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2220009820 tx=0x7f2220009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ecbf96f27ebc5 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:13.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f221ffff700 1 -- 172.21.15.36:0/2870998159 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220005e30 con 0x7f2230159250 2022-09-20T20:07:13.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f221ffff700 1 -- 172.21.15.36:0/2870998159 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2220006030 con 0x7f2230159250 2022-09-20T20:07:13.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f221ffff700 1 -- 172.21.15.36:0/2870998159 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22200063e0 con 0x7f2230159250 2022-09-20T20:07:13.552 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f2237855700 1 -- 172.21.15.36:0/2870998159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2230159250 msgr2=0x7f22300cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f2237855700 1 --2- 172.21.15.36:0/2870998159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2230159250 0x7f22300cd310 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2220009820 tx=0x7f2220009b30 comp rx=0 tx=0).stop 2022-09-20T20:07:13.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f2237855700 1 -- 172.21.15.36:0/2870998159 shutdown_connections 2022-09-20T20:07:13.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f2237855700 1 --2- 172.21.15.36:0/2870998159 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f2230159250 0x7f22300cd310 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f2237855700 1 --2- 172.21.15.36:0/2870998159 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f22300cba10 0x7f22300cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.553 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f2237855700 1 -- 172.21.15.36:0/2870998159 >> 172.21.15.36:0/2870998159 conn(0x7f22300695c0 msgr2=0x7f2230098a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:13.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f2237855700 1 -- 172.21.15.36:0/2870998159 shutdown_connections 2022-09-20T20:07:13.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.545+0000 7f2237855700 1 -- 172.21.15.36:0/2870998159 wait complete. 2022-09-20T20:07:13.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f2237855700 1 Processor -- start 2022-09-20T20:07:13.554 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f2237855700 1 -- start start 2022-09-20T20:07:13.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f2237855700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22300c9d90 0x7f22300b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.558 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f2237855700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22300cba10 0x7f22300c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f2237855700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2230159250 0x7f22301df7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f2237855700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b5550 con 0x7f22300c9d90 2022-09-20T20:07:13.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f2237855700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b4f90 con 0x7f22300cba10 2022-09-20T20:07:13.559 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f2237855700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b5270 con 0x7f2230159250 2022-09-20T20:07:13.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f22355f1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22300c9d90 0x7f22300b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f22355f1700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22300c9d90 0x7f22300b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58318/0 (socket says 172.21.15.36:58318) 2022-09-20T20:07:13.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.547+0000 7f22355f1700 1 -- 172.21.15.36:0/2998937320 learned_addr learned my addr 172.21.15.36:0/2998937320 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:13.560 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f2235df2700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2230159250 0x7f22301df7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f2234df0700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22300cba10 0x7f22300c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f22355f1700 1 -- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2230159250 msgr2=0x7f22301df7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f22355f1700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2230159250 0x7f22301df7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f22355f1700 1 -- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22300cba10 msgr2=0x7f22300c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.561 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f22355f1700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22300cba10 0x7f22300c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f22355f1700 1 -- 172.21.15.36:0/2998937320 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2220009430 con 0x7f22300c9d90 2022-09-20T20:07:13.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f2235df2700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2230159250 0x7f22301df7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-09-20T20:07:13.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f2234df0700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22300cba10 0x7f22300c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-09-20T20:07:13.562 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f22355f1700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22300c9d90 0x7f22300b4920 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2220013040 tx=0x7f2220004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:13.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f221dffb700 1 -- 172.21.15.36:0/2998937320 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220025430 con 0x7f22300c9d90 2022-09-20T20:07:13.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f221dffb700 1 -- 172.21.15.36:0/2998937320 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2220020070 con 0x7f22300c9d90 2022-09-20T20:07:13.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f2237855700 1 -- 172.21.15.36:0/2998937320 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2230069f50 con 0x7f22300c9d90 2022-09-20T20:07:13.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f221dffb700 1 -- 172.21.15.36:0/2998937320 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22200257d0 con 0x7f22300c9d90 2022-09-20T20:07:13.563 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.548+0000 7f2237855700 1 -- 172.21.15.36:0/2998937320 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22300c5210 con 0x7f22300c9d90 2022-09-20T20:07:13.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.549+0000 7f2237855700 1 -- 172.21.15.36:0/2998937320 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2230069db0 con 0x7f22300c9d90 2022-09-20T20:07:13.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.550+0000 7f221dffb700 1 -- 172.21.15.36:0/2998937320 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7f222002b020 con 0x7f22300c9d90 2022-09-20T20:07:13.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.550+0000 7f221dffb700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f2210065a50 0x7f2210067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.564 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.550+0000 7f221dffb700 1 -- 172.21.15.36:0/2998937320 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2220084000 con 0x7f22300c9d90 2022-09-20T20:07:13.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.550+0000 7f2234df0700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f2210065a50 0x7f2210067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.551+0000 7f2234df0700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f2210065a50 0x7f2210067f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22301f3ce0 tx=0x7f222c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:13.565 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.554+0000 7f221dffb700 1 -- 172.21.15.36:0/2998937320 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f2230069db0 con 0x7f22300c9d90 2022-09-20T20:07:13.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.742+0000 7f2237855700 1 -- 172.21.15.36:0/2998937320 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2230069db0 con 0x7f22300c9d90 2022-09-20T20:07:13.744 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.742+0000 7f221dffb700 1 -- 172.21.15.36:0/2998937320 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2230069db0 con 0x7f22300c9d90 2022-09-20T20:07:13.745 INFO:teuthology.orchestra.run.smithi036.stdout:pg_num: 8 2022-09-20T20:07:13.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.744+0000 7f2237855700 1 -- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f2210065a50 msgr2=0x7f2210067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.744+0000 7f2237855700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f2210065a50 0x7f2210067f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22301f3ce0 tx=0x7f222c008040 comp rx=0 tx=0).stop 2022-09-20T20:07:13.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.744+0000 7f2237855700 1 -- 172.21.15.36:0/2998937320 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22300c9d90 msgr2=0x7f22300b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.747 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.744+0000 7f2237855700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22300c9d90 0x7f22300b4920 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2220013040 tx=0x7f2220004750 comp rx=0 tx=0).stop 2022-09-20T20:07:13.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.744+0000 7f2237855700 1 -- 172.21.15.36:0/2998937320 shutdown_connections 2022-09-20T20:07:13.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.744+0000 7f2237855700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f2210065a50 0x7f2210067f00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.744+0000 7f2237855700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f2230159250 0x7f22301df7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.748 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.744+0000 7f2237855700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f22300c9d90 0x7f22300b4920 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.744+0000 7f2237855700 1 --2- 172.21.15.36:0/2998937320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22300cba10 0x7f22300c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.744+0000 7f2237855700 1 -- 172.21.15.36:0/2998937320 >> 172.21.15.36:0/2998937320 conn(0x7f22300695c0 msgr2=0x7f223006ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:13.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.745+0000 7f2237855700 1 -- 172.21.15.36:0/2998937320 shutdown_connections 2022-09-20T20:07:13.749 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.745+0000 7f2237855700 1 -- 172.21.15.36:0/2998937320 wait complete. 2022-09-20T20:07:13.759 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-09-20T20:07:13.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.881+0000 7f0e3742c700 1 Processor -- start 2022-09-20T20:07:13.883 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.881+0000 7f0e3742c700 1 -- start start 2022-09-20T20:07:13.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.882+0000 7f0e3742c700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300c8580 0x7f0e300ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.895 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.882+0000 7f0e3742c700 1 -- --> v1:172.21.15.36:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e30069e70 con 0x7f0e300ac030 2022-09-20T20:07:13.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.882+0000 7f0e3742c700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3006a150 con 0x7f0e300c8950 2022-09-20T20:07:13.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.882+0000 7f0e3742c700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300aa8f0 con 0x7f0e300c8580 2022-09-20T20:07:13.896 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.882+0000 7f0e359c9700 1 --1- >> v1:172.21.15.36:6789/0 conn(0x7f0e300ac030 0x7f0e3015e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.36:6789/0 says I am v1:172.21.15.36:37766/0 (socket says 172.21.15.36:37766) 2022-09-20T20:07:13.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.882+0000 7f0e359c9700 1 -- 172.21.15.36:0/3413902281 learned_addr learned my addr 172.21.15.36:0/3413902281 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-09-20T20:07:13.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e351c8700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300c8580 0x7f0e300ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.897 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 <== mon.0 v1:172.21.15.36:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787048336 0 0) 0x7f0e30069e70 con 0x7f0e300ac030 2022-09-20T20:07:13.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 --> v1:172.21.15.36:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e180035b0 con 0x7f0e300ac030 2022-09-20T20:07:13.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 <== mon.0 v1:172.21.15.36:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3179703761 0 0) 0x7f0e180035b0 con 0x7f0e300ac030 2022-09-20T20:07:13.898 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300c8580 msgr2=0x7f0e300ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:13.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300c8580 0x7f0e300ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 >> v1:172.21.15.115:6789/0 conn(0x7f0e300c8950 legacy=0x7f0e300696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 --> v1:172.21.15.36:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3006a430 con 0x7f0e300ac030 2022-09-20T20:07:13.899 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 <== mon.0 v1:172.21.15.36:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3393640461 0 0) 0x7f0e280019f0 con 0x7f0e300ac030 2022-09-20T20:07:13.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 >> v1:172.21.15.36:6789/0 conn(0x7f0e300ac030 legacy=0x7f0e3015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e18003a30 0x7f0e18003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.900 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300c8580 0x7f0e18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e18004d90 0x7f0e18005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300aa8f0 con 0x7f0e18003a30 2022-09-20T20:07:13.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1800bac0 con 0x7f0e18004d90 2022-09-20T20:07:13.901 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1800bc50 con 0x7f0e300c8580 2022-09-20T20:07:13.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e351c8700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e18003a30 0x7f0e18003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e359c9700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e18004d90 0x7f0e18005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e349c7700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300c8580 0x7f0e18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.902 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e351c8700 1 -- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300c8580 msgr2=0x7f0e18004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:13.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e351c8700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300c8580 0x7f0e18004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e351c8700 1 -- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e18004d90 msgr2=0x7f0e18005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:13.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e351c8700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e18004d90 0x7f0e18005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.903 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e351c8700 1 -- 172.21.15.36:0/3413902281 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180035b0 con 0x7f0e18003a30 2022-09-20T20:07:13.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e359c9700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e18004d90 0x7f0e18005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-09-20T20:07:13.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.883+0000 7f0e349c7700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300c8580 0x7f0e18004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-09-20T20:07:13.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e351c8700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e18003a30 0x7f0e18003e40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009b60 tx=0x7f0e18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d46b63625b0fc20 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:13.904 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2000e6c0 con 0x7f0e18003a30 2022-09-20T20:07:13.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2000e820 con 0x7f0e18003a30 2022-09-20T20:07:13.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e1ffff700 1 -- 172.21.15.36:0/3413902281 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2000eb90 con 0x7f0e18003a30 2022-09-20T20:07:13.905 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e3742c700 1 -- 172.21.15.36:0/3413902281 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e18003a30 msgr2=0x7f0e18003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e3742c700 1 --2- 172.21.15.36:0/3413902281 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e18003a30 0x7f0e18003e40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009b60 tx=0x7f0e18005880 comp rx=0 tx=0).stop 2022-09-20T20:07:13.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e359c9700 1 -- 172.21.15.36:0/3413902281 reap_dead start 2022-09-20T20:07:13.906 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e3742c700 1 -- 172.21.15.36:0/3413902281 shutdown_connections 2022-09-20T20:07:13.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e3742c700 1 -- 172.21.15.36:0/3413902281 >> 172.21.15.36:0/3413902281 conn(0x7f0e300ce950 msgr2=0x7f0e300ca740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:13.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e3742c700 1 -- 172.21.15.36:0/3413902281 shutdown_connections 2022-09-20T20:07:13.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.884+0000 7f0e3742c700 1 -- 172.21.15.36:0/3413902281 wait complete. 2022-09-20T20:07:13.907 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.886+0000 7f0e3742c700 1 Processor -- start 2022-09-20T20:07:13.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.886+0000 7f0e3742c700 1 -- start start 2022-09-20T20:07:13.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e3742c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e18003a30 0x7f0e301580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e3742c700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e3015f640 0x7f0e300c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.908 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e3742c700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300b0bc0 0x7f0e30159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e3742c700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300c4440 con 0x7f0e3015f640 2022-09-20T20:07:13.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e3742c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3015a240 con 0x7f0e18003a30 2022-09-20T20:07:13.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e3742c700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3015b0a0 con 0x7f0e300b0bc0 2022-09-20T20:07:13.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e349c7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e3015f640 0x7f0e300c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.909 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e349c7700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e3015f640 0x7f0e300c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58338/0 (socket says 172.21.15.36:58338) 2022-09-20T20:07:13.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e349c7700 1 -- 172.21.15.36:0/3673373638 learned_addr learned my addr 172.21.15.36:0/3673373638 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:13.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e359c9700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300b0bc0 0x7f0e30159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e351c8700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e18003a30 0x7f0e301580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.910 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e349c7700 1 -- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300b0bc0 msgr2=0x7f0e30159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e349c7700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300b0bc0 0x7f0e30159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e349c7700 1 -- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e18003a30 msgr2=0x7f0e301580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:13.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e349c7700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e18003a30 0x7f0e301580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:13.911 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e349c7700 1 -- 172.21.15.36:0/3673373638 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3015a490 con 0x7f0e3015f640 2022-09-20T20:07:13.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.887+0000 7f0e349c7700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e3015f640 0x7f0e300c3f00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00e7b0 tx=0x7f0e2c00eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:13.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.888+0000 7f0e1dffb700 1 -- 172.21.15.36:0/3673373638 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0052e0 con 0x7f0e3015f640 2022-09-20T20:07:13.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.888+0000 7f0e1dffb700 1 -- 172.21.15.36:0/3673373638 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c005440 con 0x7f0e3015f640 2022-09-20T20:07:13.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.888+0000 7f0e3742c700 1 -- 172.21.15.36:0/3673373638 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3015a770 con 0x7f0e3015f640 2022-09-20T20:07:13.912 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.888+0000 7f0e1dffb700 1 -- 172.21.15.36:0/3673373638 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0038c0 con 0x7f0e3015f640 2022-09-20T20:07:13.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.888+0000 7f0e3742c700 1 -- 172.21.15.36:0/3673373638 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3015ad30 con 0x7f0e3015f640 2022-09-20T20:07:13.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.889+0000 7f0e3742c700 1 -- 172.21.15.36:0/3673373638 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3015aef0 con 0x7f0e3015f640 2022-09-20T20:07:13.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.889+0000 7f0e1dffb700 1 -- 172.21.15.36:0/3673373638 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7f0e2c01b030 con 0x7f0e3015f640 2022-09-20T20:07:13.913 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.890+0000 7f0e1dffb700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f0e10065a80 0x7f0e10067f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:13.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.890+0000 7f0e1dffb700 1 -- 172.21.15.36:0/3673373638 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0e2c081c40 con 0x7f0e3015f640 2022-09-20T20:07:13.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.890+0000 7f0e351c8700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f0e10065a80 0x7f0e10067f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:13.914 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.890+0000 7f0e351c8700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f0e10065a80 0x7f0e10067f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0e200090b0 tx=0x7f0e20008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:13.915 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:13.894+0000 7f0e1dffb700 1 -- 172.21.15.36:0/3673373638 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f0e2c04ca60 con 0x7f0e3015f640 2022-09-20T20:07:14.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.089+0000 7f0e3742c700 1 -- 172.21.15.36:0/3673373638 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0e300663c0 con 0x7f0e3015f640 2022-09-20T20:07:14.091 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.089+0000 7f0e1dffb700 1 -- 172.21.15.36:0/3673373638 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f0e2c019090 con 0x7f0e3015f640 2022-09-20T20:07:14.091 INFO:teuthology.orchestra.run.smithi036.stdout: 2022-09-20T20:07:14.092 INFO:teuthology.orchestra.run.smithi036.stdout:[ 2022-09-20T20:07:14.092 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-09-20T20:07:14.092 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 1, 2022-09-20T20:07:14.092 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": ".mgr" 2022-09-20T20:07:14.093 INFO:teuthology.orchestra.run.smithi036.stdout: }, 2022-09-20T20:07:14.093 INFO:teuthology.orchestra.run.smithi036.stdout: { 2022-09-20T20:07:14.093 INFO:teuthology.orchestra.run.smithi036.stdout: "poolnum": 2, 2022-09-20T20:07:14.093 INFO:teuthology.orchestra.run.smithi036.stdout: "poolname": "rbd" 2022-09-20T20:07:14.094 INFO:teuthology.orchestra.run.smithi036.stdout: } 2022-09-20T20:07:14.094 INFO:teuthology.orchestra.run.smithi036.stdout:] 2022-09-20T20:07:14.095 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.091+0000 7f0e3742c700 1 -- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f0e10065a80 msgr2=0x7f0e10067f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:14.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.091+0000 7f0e3742c700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f0e10065a80 0x7f0e10067f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0e200090b0 tx=0x7f0e20008030 comp rx=0 tx=0).stop 2022-09-20T20:07:14.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.091+0000 7f0e3742c700 1 -- 172.21.15.36:0/3673373638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e3015f640 msgr2=0x7f0e300c3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:14.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.091+0000 7f0e3742c700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e3015f640 0x7f0e300c3f00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00e7b0 tx=0x7f0e2c00eb70 comp rx=0 tx=0).stop 2022-09-20T20:07:14.096 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.091+0000 7f0e3742c700 1 -- 172.21.15.36:0/3673373638 shutdown_connections 2022-09-20T20:07:14.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.092+0000 7f0e3742c700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f0e10065a80 0x7f0e10067f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:14.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.092+0000 7f0e3742c700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f0e300b0bc0 0x7f0e30159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:14.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.092+0000 7f0e3742c700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e18003a30 0x7f0e301580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:14.097 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.092+0000 7f0e3742c700 1 --2- 172.21.15.36:0/3673373638 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f0e3015f640 0x7f0e300c3f00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:14.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.092+0000 7f0e3742c700 1 -- 172.21.15.36:0/3673373638 >> 172.21.15.36:0/3673373638 conn(0x7f0e300ce950 msgr2=0x7f0e300cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:14.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.092+0000 7f0e3742c700 1 -- 172.21.15.36:0/3673373638 shutdown_connections 2022-09-20T20:07:14.098 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.092+0000 7f0e3742c700 1 -- 172.21.15.36:0/3673373638 wait complete. 2022-09-20T20:07:14.105 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-09-20T20:07:14.106 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2022-09-20T20:07:14.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.232+0000 7f18fa891700 1 Processor -- start 2022-09-20T20:07:14.234 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.232+0000 7f18fa891700 1 -- start start 2022-09-20T20:07:14.237 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18fa891700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f18f40cb370 0x7f18f415c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:14.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18fa891700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f18f4158250 0x7f18f40c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:14.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18fa891700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f41592c0 con 0x7f18f4157e80 2022-09-20T20:07:14.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18fa891700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f415c7b0 con 0x7f18f40cb370 2022-09-20T20:07:14.238 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18fa891700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f40ad9f0 con 0x7f18f4158250 2022-09-20T20:07:14.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18f37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f18f40cb370 0x7f18f415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:14.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18f37fe700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f18f40cb370 0x7f18f415c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58346/0 (socket says 172.21.15.36:58346) 2022-09-20T20:07:14.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18f37fe700 1 -- 172.21.15.36:0/1382116620 learned_addr learned my addr 172.21.15.36:0/1382116620 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:14.239 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18f8e2e700 1 --2- 172.21.15.36:0/1382116620 >> v2:172.21.15.115:3300/0 conn(0x7f18f4158250 0x7f18f40c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:14.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18f37fe700 1 -- 172.21.15.36:0/1382116620 >> v2:172.21.15.115:3300/0 conn(0x7f18f4158250 msgr2=0x7f18f40c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:14.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18f37fe700 1 --2- 172.21.15.36:0/1382116620 >> v2:172.21.15.115:3300/0 conn(0x7f18f4158250 0x7f18f40c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:14.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18f37fe700 1 -- 172.21.15.36:0/1382116620 >> v1:172.21.15.115:6789/0 conn(0x7f18f4157e80 legacy=0x7f18f40cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:14.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18f37fe700 1 -- 172.21.15.36:0/1382116620 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40c90d0 con 0x7f18f40cb370 2022-09-20T20:07:14.240 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.233+0000 7f18f8e2e700 1 --2- 172.21.15.36:0/1382116620 >> v2:172.21.15.115:3300/0 conn(0x7f18f4158250 0x7f18f40c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-09-20T20:07:14.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18f37fe700 1 --2- 172.21.15.36:0/1382116620 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f40cb370 0x7f18f415c270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18f406fd70 tx=0x7f18e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e32f047cbb0baa1 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:14.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18f2ffd700 1 -- 172.21.15.36:0/1382116620 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8004230 con 0x7f18f40cb370 2022-09-20T20:07:14.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18f2ffd700 1 -- 172.21.15.36:0/1382116620 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e8004390 con 0x7f18f40cb370 2022-09-20T20:07:14.241 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18f2ffd700 1 -- 172.21.15.36:0/1382116620 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e80036a0 con 0x7f18f40cb370 2022-09-20T20:07:14.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18fa891700 1 -- 172.21.15.36:0/1382116620 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f40cb370 msgr2=0x7f18f415c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:14.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18fa891700 1 --2- 172.21.15.36:0/1382116620 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f40cb370 0x7f18f415c270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18f406fd70 tx=0x7f18e8009bd0 comp rx=0 tx=0).stop 2022-09-20T20:07:14.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18fa891700 1 -- 172.21.15.36:0/1382116620 shutdown_connections 2022-09-20T20:07:14.242 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18fa891700 1 --2- 172.21.15.36:0/1382116620 >> v2:172.21.15.115:3300/0 conn(0x7f18f4158250 0x7f18f40c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:14.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18fa891700 1 --2- 172.21.15.36:0/1382116620 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f40cb370 0x7f18f415c270 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:14.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18fa891700 1 -- 172.21.15.36:0/1382116620 >> 172.21.15.36:0/1382116620 conn(0x7f18f406cc90 msgr2=0x7f18f40b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:14.243 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18fa891700 1 -- 172.21.15.36:0/1382116620 shutdown_connections 2022-09-20T20:07:14.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.234+0000 7f18fa891700 1 -- 172.21.15.36:0/1382116620 wait complete. 2022-09-20T20:07:14.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.235+0000 7f18fa891700 1 Processor -- start 2022-09-20T20:07:14.244 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.236+0000 7f18fa891700 1 -- start start 2022-09-20T20:07:14.250 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.236+0000 7f18fa891700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18f40cb370 0x7f18f41462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:14.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.236+0000 7f18fa891700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f4157e80 0x7f18f4156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:14.251 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.236+0000 7f18fa891700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f4158250 0x7f18f4151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:14.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.236+0000 7f18fa891700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f4157100 con 0x7f18f4158250 2022-09-20T20:07:14.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.236+0000 7f18fa891700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f41555a0 con 0x7f18f4157e80 2022-09-20T20:07:14.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.236+0000 7f18fa891700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f41ee050 con 0x7f18f40cb370 2022-09-20T20:07:14.252 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f8e2e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f4158250 0x7f18f4151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:14.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f8e2e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f4158250 0x7f18f4151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58354/0 (socket says 172.21.15.36:58354) 2022-09-20T20:07:14.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f8e2e700 1 -- 172.21.15.36:0/3076438641 learned_addr learned my addr 172.21.15.36:0/3076438641 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:14.253 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f37fe700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f4157e80 0x7f18f4156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:14.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f3fff700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18f40cb370 0x7f18f41462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:14.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f8e2e700 1 -- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18f40cb370 msgr2=0x7f18f41462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:14.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f8e2e700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18f40cb370 0x7f18f41462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:14.254 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f8e2e700 1 -- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f4157e80 msgr2=0x7f18f4156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:14.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f8e2e700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f4157e80 0x7f18f4156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:14.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f8e2e700 1 -- 172.21.15.36:0/3076438641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8009550 con 0x7f18f4158250 2022-09-20T20:07:14.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f37fe700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f4157e80 0x7f18f4156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-09-20T20:07:14.255 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f3fff700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18f40cb370 0x7f18f41462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-09-20T20:07:14.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f8e2e700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f4158250 0x7f18f4151f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00cd80 tx=0x7f18ec00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:14.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f0ff9700 1 -- 172.21.15.36:0/3076438641 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00f480 con 0x7f18f4158250 2022-09-20T20:07:14.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f0ff9700 1 -- 172.21.15.36:0/3076438641 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec00f600 con 0x7f18f4158250 2022-09-20T20:07:14.256 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18fa891700 1 -- 172.21.15.36:0/3076438641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f4156320 con 0x7f18f4158250 2022-09-20T20:07:14.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18f0ff9700 1 -- 172.21.15.36:0/3076438641 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00f9b0 con 0x7f18f4158250 2022-09-20T20:07:14.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.237+0000 7f18fa891700 1 -- 172.21.15.36:0/3076438641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f4155b20 con 0x7f18f4158250 2022-09-20T20:07:14.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.238+0000 7f18fa891700 1 -- 172.21.15.36:0/3076438641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f4155270 con 0x7f18f4158250 2022-09-20T20:07:14.257 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.239+0000 7f18f0ff9700 1 -- 172.21.15.36:0/3076438641 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7f18ec009030 con 0x7f18f4158250 2022-09-20T20:07:14.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.239+0000 7f18f0ff9700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f18d4065a50 0x7f18d4067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:14.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.239+0000 7f18f0ff9700 1 -- 172.21.15.36:0/3076438641 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f18ec083130 con 0x7f18f4158250 2022-09-20T20:07:14.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.239+0000 7f18f3fff700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f18d4065a50 0x7f18d4067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:14.258 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.240+0000 7f18f3fff700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f18d4065a50 0x7f18d4067f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18e4003fc0 tx=0x7f18e4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:14.259 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.243+0000 7f18f0ff9700 1 -- 172.21.15.36:0/3076438641 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f18ec051210 con 0x7f18f4158250 2022-09-20T20:07:14.437 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:14.436+0000 7f18fa891700 1 -- 172.21.15.36:0/3076438641 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f18f40663c0 con 0x7f18f4158250 2022-09-20T20:07:15.409 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.407+0000 7f18f0ff9700 1 -- 172.21.15.36:0/3076438641 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f18ec054830 con 0x7f18f4158250 2022-09-20T20:07:15.413 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.411+0000 7f18fa891700 1 -- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f18d4065a50 msgr2=0x7f18d4067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:15.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.411+0000 7f18fa891700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f18d4065a50 0x7f18d4067f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18e4003fc0 tx=0x7f18e4009040 comp rx=0 tx=0).stop 2022-09-20T20:07:15.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.412+0000 7f18fa891700 1 -- 172.21.15.36:0/3076438641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f4158250 msgr2=0x7f18f4151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:15.414 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.412+0000 7f18fa891700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f4158250 0x7f18f4151f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00cd80 tx=0x7f18ec00a560 comp rx=0 tx=0).stop 2022-09-20T20:07:15.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.412+0000 7f18fa891700 1 -- 172.21.15.36:0/3076438641 shutdown_connections 2022-09-20T20:07:15.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.412+0000 7f18fa891700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f18d4065a50 0x7f18d4067f00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:15.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.412+0000 7f18fa891700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f18f4158250 0x7f18f4151f30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:15.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.412+0000 7f18fa891700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f18f40cb370 0x7f18f41462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:15.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.412+0000 7f18fa891700 1 --2- 172.21.15.36:0/3076438641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f4157e80 0x7f18f4156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:15.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.412+0000 7f18fa891700 1 -- 172.21.15.36:0/3076438641 >> 172.21.15.36:0/3076438641 conn(0x7f18f406cc90 msgr2=0x7f18f40ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:15.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.412+0000 7f18fa891700 1 -- 172.21.15.36:0/3076438641 shutdown_connections 2022-09-20T20:07:15.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.412+0000 7f18fa891700 1 -- 172.21.15.36:0/3076438641 wait complete. 2022-09-20T20:07:15.418 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_metadata' created 2022-09-20T20:07:15.431 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2022-09-20T20:07:15.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.567+0000 7f4d782e3700 1 Processor -- start 2022-09-20T20:07:15.569 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.567+0000 7f4d782e3700 1 -- start start 2022-09-20T20:07:15.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.568+0000 7f4d782e3700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 0x7f4d700cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:15.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.568+0000 7f4d782e3700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d700ac6d0 0x7f4d700aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:15.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.568+0000 7f4d782e3700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d700adaf0 con 0x7f4d700ad5a0 2022-09-20T20:07:15.573 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.568+0000 7f4d782e3700 1 -- --> v2:172.21.15.36:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d700aced0 con 0x7f4d700ac6d0 2022-09-20T20:07:15.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.568+0000 7f4d782e3700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700ad1b0 con 0x7f4d700ab2f0 2022-09-20T20:07:15.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.568+0000 7f4d76880700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d700ac6d0 0x7f4d700aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:15.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.568+0000 7f4d76880700 1 --2- >> v2:172.21.15.36:3300/0 conn(0x7f4d700ac6d0 0x7f4d700aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58362/0 (socket says 172.21.15.36:58362) 2022-09-20T20:07:15.574 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.568+0000 7f4d76880700 1 -- 172.21.15.36:0/2180086230 learned_addr learned my addr 172.21.15.36:0/2180086230 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:15.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d76880700 1 -- 172.21.15.36:0/2180086230 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 msgr2=0x7f4d700cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:15.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d7607f700 1 --2- 172.21.15.36:0/2180086230 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 0x7f4d700cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:15.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d76880700 1 --2- 172.21.15.36:0/2180086230 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 0x7f4d700cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:15.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d76880700 1 -- 172.21.15.36:0/2180086230 >> v1:172.21.15.115:6789/0 conn(0x7f4d700ad5a0 legacy=0x7f4d700ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:15.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d76880700 1 -- 172.21.15.36:0/2180086230 --> v2:172.21.15.36:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d700add40 con 0x7f4d700ac6d0 2022-09-20T20:07:15.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d76880700 1 --2- 172.21.15.36:0/2180086230 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 0x7f4d700aec50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4d68009e50 tx=0x7f4d6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5812fce1ace62045 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:15.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d7507d700 1 -- 172.21.15.36:0/2180086230 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800e070 con 0x7f4d700ac6d0 2022-09-20T20:07:15.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d7507d700 1 -- 172.21.15.36:0/2180086230 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d68003a20 con 0x7f4d700ac6d0 2022-09-20T20:07:15.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d7507d700 1 -- 172.21.15.36:0/2180086230 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d68003dd0 con 0x7f4d700ac6d0 2022-09-20T20:07:15.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d782e3700 1 -- 172.21.15.36:0/2180086230 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 msgr2=0x7f4d700aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:15.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.569+0000 7f4d782e3700 1 --2- 172.21.15.36:0/2180086230 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 0x7f4d700aec50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4d68009e50 tx=0x7f4d6800b330 comp rx=0 tx=0).stop 2022-09-20T20:07:15.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.570+0000 7f4d782e3700 1 -- 172.21.15.36:0/2180086230 shutdown_connections 2022-09-20T20:07:15.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.570+0000 7f4d782e3700 1 --2- 172.21.15.36:0/2180086230 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 0x7f4d700aec50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:15.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.570+0000 7f4d782e3700 1 --2- 172.21.15.36:0/2180086230 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 0x7f4d700cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:15.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.570+0000 7f4d782e3700 1 -- 172.21.15.36:0/2180086230 >> 172.21.15.36:0/2180086230 conn(0x7f4d7006a6d0 msgr2=0x7f4d700ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:15.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.570+0000 7f4d782e3700 1 -- 172.21.15.36:0/2180086230 shutdown_connections 2022-09-20T20:07:15.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.570+0000 7f4d782e3700 1 -- 172.21.15.36:0/2180086230 wait complete. 2022-09-20T20:07:15.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.571+0000 7f4d782e3700 1 Processor -- start 2022-09-20T20:07:15.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.571+0000 7f4d782e3700 1 -- start start 2022-09-20T20:07:15.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.571+0000 7f4d782e3700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 0x7f4d70143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:15.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d782e3700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 0x7f4d700c2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:15.583 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d782e3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d700ad5a0 0x7f4d700c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:15.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d782e3700 1 -- --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d701f1570 con 0x7f4d700ac6d0 2022-09-20T20:07:15.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d782e3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d70147040 con 0x7f4d700ad5a0 2022-09-20T20:07:15.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d782e3700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d701f1a00 con 0x7f4d700ab2f0 2022-09-20T20:07:15.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7587e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 0x7f4d700c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:15.584 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7587e700 1 --2- >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 0x7f4d700c2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.36:3300/0 says I am v2:172.21.15.36:58366/0 (socket says 172.21.15.36:58366) 2022-09-20T20:07:15.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7587e700 1 -- 172.21.15.36:0/1548109514 learned_addr learned my addr 172.21.15.36:0/1548109514 (peer_addr_for_me v2:172.21.15.36:0/0) 2022-09-20T20:07:15.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7607f700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 0x7f4d70143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:15.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d76880700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d700ad5a0 0x7f4d700c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:15.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7587e700 1 -- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 msgr2=0x7f4d70143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:15.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7587e700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 0x7f4d70143790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:15.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7587e700 1 -- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d700ad5a0 msgr2=0x7f4d700c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:15.586 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7587e700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d700ad5a0 0x7f4d700c26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:15.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7587e700 1 -- 172.21.15.36:0/1548109514 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d68009b30 con 0x7f4d700ac6d0 2022-09-20T20:07:15.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d76880700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d700ad5a0 0x7f4d700c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-09-20T20:07:15.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7607f700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 0x7f4d70143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-09-20T20:07:15.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d7587e700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 0x7f4d700c2190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00c500 tx=0x7f4d6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:15.587 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d5effd700 1 -- 172.21.15.36:0/1548109514 <== mon.0 v2:172.21.15.36:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0108a0 con 0x7f4d700ac6d0 2022-09-20T20:07:15.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d5effd700 1 -- 172.21.15.36:0/1548109514 <== mon.0 v2:172.21.15.36:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c010a00 con 0x7f4d700ac6d0 2022-09-20T20:07:15.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d782e3700 1 -- 172.21.15.36:0/1548109514 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d701f0c40 con 0x7f4d700ac6d0 2022-09-20T20:07:15.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d5effd700 1 -- 172.21.15.36:0/1548109514 <== mon.0 v2:172.21.15.36:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c010db0 con 0x7f4d700ac6d0 2022-09-20T20:07:15.588 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.572+0000 7f4d782e3700 1 -- 172.21.15.36:0/1548109514 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d701f1200 con 0x7f4d700ac6d0 2022-09-20T20:07:15.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.573+0000 7f4d782e3700 1 -- 172.21.15.36:0/1548109514 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d700663c0 con 0x7f4d700ac6d0 2022-09-20T20:07:15.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.574+0000 7f4d5effd700 1 -- 172.21.15.36:0/1548109514 <== mon.0 v2:172.21.15.36:3300/0 4 ==== mgrmap(e 5) v1 ==== 83910+0+0 (secure 0 0 0) 0x7f4d6c022070 con 0x7f4d700ac6d0 2022-09-20T20:07:15.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.574+0000 7f4d5effd700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f4d50065a50 0x7f4d50067f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:15.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.574+0000 7f4d5effd700 1 -- 172.21.15.36:0/1548109514 <== mon.0 v2:172.21.15.36:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f4d6c0847e0 con 0x7f4d700ac6d0 2022-09-20T20:07:15.589 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.575+0000 7f4d7607f700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f4d50065a50 0x7f4d50067f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:15.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.575+0000 7f4d7607f700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f4d50065a50 0x7f4d50067f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000a9b0 tx=0x7f4d60008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:15.590 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.578+0000 7f4d5effd700 1 -- 172.21.15.36:0/1548109514 <== mon.0 v2:172.21.15.36:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170563 (secure 0 0 0) 0x7f4d6c052870 con 0x7f4d700ac6d0 2022-09-20T20:07:15.769 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:15.768+0000 7f4d782e3700 1 -- 172.21.15.36:0/1548109514 --> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f4d701f16b0 con 0x7f4d700ac6d0 2022-09-20T20:07:16.080 DEBUG:teuthology.orchestra.run.smithi036:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-09-20T20:07:16.083 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-09-20T20:07:16.408 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.406+0000 7f4d5effd700 1 -- 172.21.15.36:0/1548109514 <== mon.0 v2:172.21.15.36:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) v1 ==== 171+0+0 (secure 0 0 0) 0x7f4d6c0200d0 con 0x7f4d700ac6d0 2022-09-20T20:07:16.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.412+0000 7f4d782e3700 1 -- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f4d50065a50 msgr2=0x7f4d50067f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:16.415 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.412+0000 7f4d782e3700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f4d50065a50 0x7f4d50067f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000a9b0 tx=0x7f4d60008040 comp rx=0 tx=0).stop 2022-09-20T20:07:16.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.412+0000 7f4d782e3700 1 -- 172.21.15.36:0/1548109514 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 msgr2=0x7f4d700c2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:16.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.412+0000 7f4d782e3700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 0x7f4d700c2190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00c500 tx=0x7f4d6c00c8c0 comp rx=0 tx=0).stop 2022-09-20T20:07:16.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.412+0000 7f4d782e3700 1 -- 172.21.15.36:0/1548109514 shutdown_connections 2022-09-20T20:07:16.416 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.412+0000 7f4d782e3700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:6832/99436,v1:172.21.15.115:6833/99436] conn(0x7f4d50065a50 0x7f4d50067f00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:16.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.412+0000 7f4d782e3700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d700ad5a0 0x7f4d700c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:16.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.412+0000 7f4d782e3700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f4d700ab2f0 0x7f4d70143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:16.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.412+0000 7f4d782e3700 1 --2- 172.21.15.36:0/1548109514 >> [v2:172.21.15.36:3300/0,v1:172.21.15.36:6789/0] conn(0x7f4d700ac6d0 0x7f4d700c2190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:16.417 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.412+0000 7f4d782e3700 1 -- 172.21.15.36:0/1548109514 >> 172.21.15.36:0/1548109514 conn(0x7f4d7006a6d0 msgr2=0x7f4d70157bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:16.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.414+0000 7f4d782e3700 1 -- 172.21.15.36:0/1548109514 shutdown_connections 2022-09-20T20:07:16.418 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.415+0000 7f4d782e3700 1 -- 172.21.15.36:0/1548109514 wait complete. 2022-09-20T20:07:16.424 INFO:teuthology.orchestra.run.smithi036.stderr:pool 'cephfs_data' created 2022-09-20T20:07:16.439 DEBUG:teuthology.orchestra.run.smithi036:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-09-20T20:07:16.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.570+0000 7f1635735700 1 Processor -- start 2022-09-20T20:07:16.572 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.570+0000 7f1635735700 1 -- start start 2022-09-20T20:07:16.575 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.571+0000 7f1635735700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f163006fa50 0x7f163006cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:16.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.571+0000 7f1635735700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f163015f8f0 0x7f163015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-09-20T20:07:16.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.571+0000 7f1635735700 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163015fd00 con 0x7f163006fe20 2022-09-20T20:07:16.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.571+0000 7f1635735700 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163015f460 con 0x7f163006fa50 2022-09-20T20:07:16.576 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.571+0000 7f1635735700 1 -- --> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163015ee50 con 0x7f163015f8f0 2022-09-20T20:07:16.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.571+0000 7f162effd700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f163006fa50 0x7f163006cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:16.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162f7fe700 1 --2- >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f163015f8f0 0x7f163015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-09-20T20:07:16.577 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162e7fc700 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f163006fe20 0x7f16300ca130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.36:48446/0 (socket says 172.21.15.36:48446) 2022-09-20T20:07:16.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162effd700 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f163006fa50 0x7f163006cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.36:51988/0 (socket says 172.21.15.36:51988) 2022-09-20T20:07:16.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162e7fc700 1 -- 172.21.15.36:0/1585129976 learned_addr learned my addr 172.21.15.36:0/1585129976 (peer_addr_for_me v1:172.21.15.36:0/0) 2022-09-20T20:07:16.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162effd700 1 -- 172.21.15.36:0/1585129976 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f163015f8f0 msgr2=0x7f163015ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-09-20T20:07:16.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162effd700 1 --2- 172.21.15.36:0/1585129976 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f163015f8f0 0x7f163015ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:16.578 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162effd700 1 -- 172.21.15.36:0/1585129976 >> v1:172.21.15.115:6789/0 conn(0x7f163006fe20 legacy=0x7f16300ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:16.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162effd700 1 -- 172.21.15.36:0/1585129976 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163006d360 con 0x7f163006fa50 2022-09-20T20:07:16.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162dffb700 1 -- 172.21.15.36:0/1585129976 <== mon.1 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289526009 0 0) 0x7f1624001110 con 0x7f163006fe20 2022-09-20T20:07:16.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162f7fe700 1 --2- 172.21.15.36:0/1585129976 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f163015f8f0 0x7f163015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-09-20T20:07:16.579 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162effd700 1 --2- 172.21.15.36:0/1585129976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f163006fa50 0x7f163006cd80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1620009840 tx=0x7f1620009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db7b80f5a4be596 server_cookie=0 in_seq=0 out_seq=0 2022-09-20T20:07:16.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162dffb700 1 -- 172.21.15.36:0/1585129976 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620011830 con 0x7f163006fa50 2022-09-20T20:07:16.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162dffb700 1 -- 172.21.15.36:0/1585129976 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1620011990 con 0x7f163006fa50 2022-09-20T20:07:16.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.572+0000 7f162dffb700 1 -- 172.21.15.36:0/1585129976 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620011d20 con 0x7f163006fa50 2022-09-20T20:07:16.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.573+0000 7f1635735700 1 -- 172.21.15.36:0/1585129976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f163006fa50 msgr2=0x7f163006cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-09-20T20:07:16.580 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.573+0000 7f1635735700 1 --2- 172.21.15.36:0/1585129976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f163006fa50 0x7f163006cd80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1620009840 tx=0x7f1620009b50 comp rx=0 tx=0).stop 2022-09-20T20:07:16.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.573+0000 7f1635735700 1 -- 172.21.15.36:0/1585129976 shutdown_connections 2022-09-20T20:07:16.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.573+0000 7f1635735700 1 --2- 172.21.15.36:0/1585129976 >> [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] conn(0x7f163015f8f0 0x7f163015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:16.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.573+0000 7f1635735700 1 --2- 172.21.15.36:0/1585129976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f163006fa50 0x7f163006cd80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-09-20T20:07:16.581 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.573+0000 7f1635735700 1 -- 172.21.15.36:0/1585129976 >> 172.21.15.36:0/1585129976 conn(0x7f16300cde30 msgr2=0x7f16300ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-09-20T20:07:16.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.573+0000 7f1635735700 1 -- 172.21.15.36:0/1585129976 shutdown_connections 2022-09-20T20:07:16.582 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.573+0000 7f1635735700 1 -- 172.21.15.36:0/1585129976 wait complete. 2022-09-20T20:07:16.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.574+0000 7f1635735700 1 Processor -- start 2022-09-20T20:07:16.585 INFO:teuthology.orchestra.run.smithi036.stderr:2022-09-20T20:07:16.574+0000 7f1635735700 1 -- start start 2022-09-20T20:07:16.585 INFO: