2022-11-27T15:03:03.613 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-27T15:03:03.614 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} --verbose --archive /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088602 --name yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi -- /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088602/orig.config.yaml 2022-11-27T15:03:03.681 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-11-27T15:03:03.765 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088602 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} email: ceph-qa@ceph.io first_in_suite: false job_id: '7088602' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - MDS_FAILED - MDS_DAMAGE sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 sleep_before_teardown: 0 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOFj8mbVZguKfonD5ijtA/JHmS+dW9gjdQiLWKApgDnyPZ7ExrXC4MKJFBpoKDqeL64+yGbw5vxUIDCsIstSCXhzXoojsVeRibEQZu7hJidMK/k6Ay8PDamRs3NYHlXgl7vsAST9F9qvdbIcbyzUzjKYcNGjRQ/WbrSsye46t1HSJuNvkcW0+P8U5W1wlNqL6i4iI3qU8ax7l+hKqCrUzEBK4MLa/OUSZancBHi5wpF3Un9SH+bcyAT3pmEUkSmOqTQSipDpCiL7yWLda6PSv0tH2WJc7Ombv+iWgVxbZ72vmX187DX9hF5i13ljzI+nJzkzWMyA/4o4PQYcQ7TphCi4WUOd+8KzToAmGXgiXtMx/bcJClOoL1kwY7P9u+vPC3ChUvyClvOliCvIjbdl3q/HrZ/i41PkBrIiUXU6U9hE79hcCi33gd95/Jiy3BHjLfI3v9hpdFdURAxxEpNvsFm/8x75FSpWR2oq+lv5oJFDvGVR+3+pYWXDAhKjd1U20= smithi085.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+owsi1HBloX+kA4jlHQiMLnh5+1NeWWDubHWHp0KFXTwr9qTixFX2ps2QQ+6yset+FnELQh7Rh3afVLA6BnoKv8FrNq1UbQ+19/RNZaAIGdCilObbYbtCB5P90EuNnv/ZXc5R7qZXEDeTZKLQ70JChdbW6Agj+rpNYUDPm7aOOoT6WzW6BZJv2Xq7VpVH0tCq+G7l+7uQgSr0TOPsgpWEh7MssLJy/WrsRJTogCPRd2y6Onlw5gFKbTuul/13Usc0NqBzHok08MjoLdzW81L7qbDjVWL+LAf0Abg0mLvKywQIJrUslgiHkrQRaknQ6T3ntRwaaQDMqcM1UE0rC0leFFSMbVGGexKW3eNdGGkv8NsRzajmLtF62DO2Cjk+ZdP1awEuTWh00+TiildRt0R8nKmKdPTUZH3PqErgOlDImQLKmix/8B8RBP21gta3up6QdMXjEVUbbeEwk0wLsIyKyF65SKvAKqPw4cQvuUBAWmBdnH8/E8AphxbtR42J5Kc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_recovery_pool teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-23_17:24:26 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23126 2022-11-27T15:03:03.765 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa; will attempt to use it 2022-11-27T15:03:03.766 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa/tasks 2022-11-27T15:03:03.767 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-11-27T15:03:03.772 INFO:teuthology.task.internal:Checking packages... 2022-11-27T15:03:03.807 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '697b0d99d9a52df92b20c8e053dcd46c75f364a9' 2022-11-27T15:03:03.808 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T15:03:03.808 INFO:teuthology.packaging:ref: None 2022-11-27T15:03:03.809 INFO:teuthology.packaging:tag: None 2022-11-27T15:03:03.809 INFO:teuthology.packaging:branch: main 2022-11-27T15:03:03.809 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T15:03:03.810 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=main 2022-11-27T15:03:04.069 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-982.g2bb1aa26 2022-11-27T15:03:04.070 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-11-27T15:03:04.083 INFO:teuthology.task.internal:no buildpackages task found 2022-11-27T15:03:04.083 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-11-27T15:03:04.098 INFO:teuthology.task.internal:Saving configuration 2022-11-27T15:03:04.136 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-11-27T15:03:04.148 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-27T15:03:04.178 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi035.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 14:58:19.158139', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD/rR7Y5/78/CwbVwBQB8eqLGBunZt21M4b5z83JjEjXa2s663ocF9gefsvqQ0p1BVl0q7jgUP8WIrimv+RTYpg='} 2022-11-27T15:03:04.209 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi085.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 14:58:19.156473', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrd74n3BHSI61eEKu/z71ER2ppTPKkI3KPGgHv0930B9JWdAIC1TzAEVNqLCxUPzCF+G1N5EPwhUwF/lyLMCkM='} 2022-11-27T15:03:04.212 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-11-27T15:03:04.225 INFO:teuthology.task.internal:roles: ubuntu@smithi035.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-11-27T15:03:04.226 INFO:teuthology.task.internal:roles: ubuntu@smithi085.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-11-27T15:03:04.226 INFO:teuthology.run_tasks:Running task console_log... 2022-11-27T15:03:04.318 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3565dd6840>, signals=[15]) 2022-11-27T15:03:04.320 INFO:teuthology.run_tasks:Running task internal.connect... 2022-11-27T15:03:04.331 INFO:teuthology.task.internal:Opening connections... 2022-11-27T15:03:04.332 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi035.front.sepia.ceph.com 2022-11-27T15:03:04.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T15:03:04.445 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi085.front.sepia.ceph.com 2022-11-27T15:03:04.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T15:03:04.546 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-11-27T15:03:04.554 DEBUG:teuthology.orchestra.run.smithi035:> uname -m 2022-11-27T15:03:04.578 INFO:teuthology.orchestra.run.smithi035.stdout:x86_64 2022-11-27T15:03:04.579 DEBUG:teuthology.orchestra.run.smithi035:> cat /etc/os-release 2022-11-27T15:03:04.640 INFO:teuthology.orchestra.run.smithi035.stdout:NAME="Red Hat Enterprise Linux" 2022-11-27T15:03:04.640 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION="8.6 (Ootpa)" 2022-11-27T15:03:04.640 INFO:teuthology.orchestra.run.smithi035.stdout:ID="rhel" 2022-11-27T15:03:04.641 INFO:teuthology.orchestra.run.smithi035.stdout:ID_LIKE="fedora" 2022-11-27T15:03:04.641 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION_ID="8.6" 2022-11-27T15:03:04.641 INFO:teuthology.orchestra.run.smithi035.stdout:PLATFORM_ID="platform:el8" 2022-11-27T15:03:04.641 INFO:teuthology.orchestra.run.smithi035.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-11-27T15:03:04.641 INFO:teuthology.orchestra.run.smithi035.stdout:ANSI_COLOR="0;31" 2022-11-27T15:03:04.642 INFO:teuthology.orchestra.run.smithi035.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-11-27T15:03:04.642 INFO:teuthology.orchestra.run.smithi035.stdout:HOME_URL="https://www.redhat.com/" 2022-11-27T15:03:04.642 INFO:teuthology.orchestra.run.smithi035.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-11-27T15:03:04.642 INFO:teuthology.orchestra.run.smithi035.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T15:03:04.643 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:03:04.643 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T15:03:04.643 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-11-27T15:03:04.643 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-11-27T15:03:04.644 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-11-27T15:03:04.644 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server 2022-11-27T15:03:04.666 DEBUG:teuthology.orchestra.run.smithi085:> uname -m 2022-11-27T15:03:04.686 INFO:teuthology.orchestra.run.smithi085.stdout:x86_64 2022-11-27T15:03:04.687 DEBUG:teuthology.orchestra.run.smithi085:> cat /etc/os-release 2022-11-27T15:03:04.746 INFO:teuthology.orchestra.run.smithi085.stdout:NAME="Red Hat Enterprise Linux" 2022-11-27T15:03:04.746 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION="8.6 (Ootpa)" 2022-11-27T15:03:04.746 INFO:teuthology.orchestra.run.smithi085.stdout:ID="rhel" 2022-11-27T15:03:04.747 INFO:teuthology.orchestra.run.smithi085.stdout:ID_LIKE="fedora" 2022-11-27T15:03:04.747 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION_ID="8.6" 2022-11-27T15:03:04.747 INFO:teuthology.orchestra.run.smithi085.stdout:PLATFORM_ID="platform:el8" 2022-11-27T15:03:04.748 INFO:teuthology.orchestra.run.smithi085.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-11-27T15:03:04.748 INFO:teuthology.orchestra.run.smithi085.stdout:ANSI_COLOR="0;31" 2022-11-27T15:03:04.748 INFO:teuthology.orchestra.run.smithi085.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-11-27T15:03:04.748 INFO:teuthology.orchestra.run.smithi085.stdout:HOME_URL="https://www.redhat.com/" 2022-11-27T15:03:04.748 INFO:teuthology.orchestra.run.smithi085.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-11-27T15:03:04.749 INFO:teuthology.orchestra.run.smithi085.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T15:03:04.749 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:03:04.749 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T15:03:04.749 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-11-27T15:03:04.750 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-11-27T15:03:04.750 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-11-27T15:03:04.750 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2022-11-27T15:03:04.770 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-11-27T15:03:04.830 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-11-27T15:03:04.848 INFO:teuthology.task.internal:Checking for old test directory... 2022-11-27T15:03:04.849 DEBUG:teuthology.orchestra.run.smithi035:> test '!' -e /home/ubuntu/cephtest 2022-11-27T15:03:04.853 DEBUG:teuthology.orchestra.run.smithi085:> test '!' -e /home/ubuntu/cephtest 2022-11-27T15:03:04.873 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-11-27T15:03:04.885 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-11-27T15:03:04.886 DEBUG:teuthology.orchestra.run.smithi035:> test -z $(ls -A /var/lib/ceph) 2022-11-27T15:03:04.914 DEBUG:teuthology.orchestra.run.smithi085:> test -z $(ls -A /var/lib/ceph) 2022-11-27T15:03:04.946 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-11-27T15:03:05.010 INFO:teuthology.run_tasks:Running task kernel... 2022-11-27T15:03:05.031 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2022-11-27T15:03:05.032 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2022-11-27T15:03:05.032 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2022-11-27T15:03:05.033 DEBUG:teuthology.orchestra.run.smithi035:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T15:03:05.034 DEBUG:teuthology.orchestra.run.smithi085:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T15:03:05.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T15:03:05.054 DEBUG:teuthology.orchestra.run.smithi085:> uname -r 2022-11-27T15:03:05.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T15:03:05.056 DEBUG:teuthology.orchestra.run.smithi035:> uname -r 2022-11-27T15:03:05.113 INFO:teuthology.orchestra.run.smithi085.stdout:4.18.0-372.9.1.el8.x86_64 2022-11-27T15:03:05.114 INFO:teuthology.task.kernel:Running kernel on smithi085: 4.18.0-372.9.1.el8.x86_64 2022-11-27T15:03:05.114 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum install -y kernel 2022-11-27T15:03:05.116 INFO:teuthology.orchestra.run.smithi035.stdout:4.18.0-372.9.1.el8.x86_64 2022-11-27T15:03:05.116 INFO:teuthology.task.kernel:Running kernel on smithi035: 4.18.0-372.9.1.el8.x86_64 2022-11-27T15:03:05.116 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum install -y kernel 2022-11-27T15:03:06.096 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2022-11-27T15:03:06.096 INFO:teuthology.orchestra.run.smithi085.stdout:Unable to read consumer identity 2022-11-27T15:03:06.209 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:03:06.209 INFO:teuthology.orchestra.run.smithi085.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T15:03:06.210 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:03:06.323 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2022-11-27T15:03:06.324 INFO:teuthology.orchestra.run.smithi035.stdout:Unable to read consumer identity 2022-11-27T15:03:06.453 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:03:06.453 INFO:teuthology.orchestra.run.smithi035.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T15:03:06.454 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:03:10.109 INFO:teuthology.orchestra.run.smithi085.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2022-11-27T15:03:11.476 INFO:teuthology.orchestra.run.smithi035.stdout:Copr repo for python3-asyncssh owned by ceph 920 B/s | 3.5 kB 00:03 2022-11-27T15:03:11.617 INFO:teuthology.orchestra.run.smithi085.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 13 MB 00:01 2022-11-27T15:03:13.044 INFO:teuthology.orchestra.run.smithi035.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 13 MB 00:01 2022-11-27T15:03:14.584 INFO:teuthology.orchestra.run.smithi085.stdout:lab-extras 353 kB/s | 24 kB 00:00 2022-11-27T15:03:15.255 INFO:teuthology.orchestra.run.smithi085.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-11-27T15:03:15.308 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2022-11-27T15:03:15.308 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2022-11-27T15:03:15.309 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2022-11-27T15:03:15.450 DEBUG:teuthology.orchestra.run.smithi085:> echo no | sudo yum reinstall kernel || true 2022-11-27T15:03:15.831 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2022-11-27T15:03:15.831 INFO:teuthology.orchestra.run.smithi085.stdout:Unable to read consumer identity 2022-11-27T15:03:15.884 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:03:15.884 INFO:teuthology.orchestra.run.smithi085.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T15:03:15.885 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:03:16.123 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 03:03:14 PM UTC. 2022-11-27T15:03:16.164 INFO:teuthology.orchestra.run.smithi085.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T15:03:16.209 INFO:teuthology.orchestra.run.smithi035.stdout:lab-extras 434 kB/s | 24 kB 00:00 2022-11-27T15:03:16.215 INFO:teuthology.orchestra.run.smithi085.stderr:Error: No packages marked for reinstall. 2022-11-27T15:03:16.258 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum reinstall -y kernel || true 2022-11-27T15:03:16.590 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2022-11-27T15:03:16.591 INFO:teuthology.orchestra.run.smithi085.stdout:Unable to read consumer identity 2022-11-27T15:03:16.642 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:03:16.643 INFO:teuthology.orchestra.run.smithi085.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T15:03:16.643 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:03:16.876 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 03:03:14 PM UTC. 2022-11-27T15:03:16.916 INFO:teuthology.orchestra.run.smithi085.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T15:03:16.937 INFO:teuthology.orchestra.run.smithi035.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-11-27T15:03:16.958 INFO:teuthology.orchestra.run.smithi085.stderr:Error: No packages marked for reinstall. 2022-11-27T15:03:16.990 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2022-11-27T15:03:16.991 INFO:teuthology.orchestra.run.smithi035.stdout:Nothing to do. 2022-11-27T15:03:16.991 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2022-11-27T15:03:17.004 DEBUG:teuthology.orchestra.run.smithi085:> rpm -q kernel --last | head -n 1 2022-11-27T15:03:17.083 INFO:teuthology.orchestra.run.smithi085.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-11-27T15:03:17.085 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-11-27T15:03:17.086 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T15:03:17.086 DEBUG:teuthology.parallel:result is None 2022-11-27T15:03:17.138 DEBUG:teuthology.orchestra.run.smithi035:> echo no | sudo yum reinstall kernel || true 2022-11-27T15:03:17.490 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2022-11-27T15:03:17.491 INFO:teuthology.orchestra.run.smithi035.stdout:Unable to read consumer identity 2022-11-27T15:03:17.543 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:03:17.543 INFO:teuthology.orchestra.run.smithi035.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T15:03:17.544 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:03:17.797 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:00:01 ago on Sun 27 Nov 2022 03:03:16 PM UTC. 2022-11-27T15:03:17.851 INFO:teuthology.orchestra.run.smithi035.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T15:03:17.884 INFO:teuthology.orchestra.run.smithi035.stderr:Error: No packages marked for reinstall. 2022-11-27T15:03:17.932 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum reinstall -y kernel || true 2022-11-27T15:03:18.283 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2022-11-27T15:03:18.283 INFO:teuthology.orchestra.run.smithi035.stdout:Unable to read consumer identity 2022-11-27T15:03:18.332 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:03:18.333 INFO:teuthology.orchestra.run.smithi035.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T15:03:18.333 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:03:18.579 INFO:teuthology.orchestra.run.smithi035.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 03:03:16 PM UTC. 2022-11-27T15:03:18.623 INFO:teuthology.orchestra.run.smithi035.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T15:03:18.651 INFO:teuthology.orchestra.run.smithi035.stderr:Error: No packages marked for reinstall. 2022-11-27T15:03:18.698 DEBUG:teuthology.orchestra.run.smithi035:> rpm -q kernel --last | head -n 1 2022-11-27T15:03:18.781 INFO:teuthology.orchestra.run.smithi035.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-11-27T15:03:18.783 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-11-27T15:03:18.783 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T15:03:18.784 DEBUG:teuthology.parallel:result is None 2022-11-27T15:03:18.784 INFO:teuthology.run_tasks:Running task internal.base... 2022-11-27T15:03:18.806 INFO:teuthology.task.internal:Creating test directory... 2022-11-27T15:03:18.808 DEBUG:teuthology.orchestra.run.smithi035:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T15:03:18.811 DEBUG:teuthology.orchestra.run.smithi085:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T15:03:18.832 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-11-27T15:03:18.841 INFO:teuthology.run_tasks:Running task internal.archive... 2022-11-27T15:03:18.854 INFO:teuthology.task.internal:Creating archive directory... 2022-11-27T15:03:18.855 DEBUG:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T15:03:18.868 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T15:03:18.907 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-11-27T15:03:18.916 INFO:teuthology.task.internal:Enabling coredump saving... 2022-11-27T15:03:18.917 DEBUG:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T15:03:18.940 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T15:03:18.977 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T15:03:18.989 INFO:teuthology.orchestra.run.smithi035.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T15:03:18.999 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T15:03:19.005 INFO:teuthology.orchestra.run.smithi035.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T15:03:19.006 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-11-27T15:03:19.020 INFO:teuthology.task.internal:Configuring sudo... 2022-11-27T15:03:19.021 DEBUG:teuthology.orchestra.run.smithi035:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T15:03:19.024 DEBUG:teuthology.orchestra.run.smithi085:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T15:03:19.072 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-11-27T15:03:19.086 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-11-27T15:03:19.086 DEBUG:teuthology.orchestra.run.smithi035:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T15:03:19.092 DEBUG:teuthology.orchestra.run.smithi085:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T15:03:19.131 DEBUG:teuthology.orchestra.run.smithi035:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T15:03:19.196 DEBUG:teuthology.orchestra.run.smithi035:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T15:03:19.246 DEBUG:teuthology.orchestra.run.smithi035:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T15:03:19.334 DEBUG:teuthology.orchestra.run.smithi035:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T15:03:19.374 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:03:19.375 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T15:03:19.441 DEBUG:teuthology.orchestra.run.smithi085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T15:03:19.491 DEBUG:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T15:03:19.544 DEBUG:teuthology.orchestra.run.smithi085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T15:03:19.632 DEBUG:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T15:03:19.675 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:03:19.675 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T15:03:19.745 DEBUG:teuthology.orchestra.run.smithi035:> sudo service rsyslog restart 2022-11-27T15:03:19.750 DEBUG:teuthology.orchestra.run.smithi085:> sudo service rsyslog restart 2022-11-27T15:03:19.809 INFO:teuthology.orchestra.run.smithi035.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T15:03:19.848 INFO:teuthology.orchestra.run.smithi085.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T15:03:20.316 INFO:teuthology.run_tasks:Running task internal.timer... 2022-11-27T15:03:20.337 INFO:teuthology.task.internal:Starting timer... 2022-11-27T15:03:20.338 INFO:teuthology.run_tasks:Running task pcp... 2022-11-27T15:03:20.358 INFO:teuthology.run_tasks:Running task selinux... 2022-11-27T15:03:20.373 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2022-11-27T15:03:20.374 DEBUG:teuthology.orchestra.run.smithi035:> sudo service auditd rotate 2022-11-27T15:03:20.437 INFO:teuthology.orchestra.run.smithi035.stdout:Rotating logs: 2022-11-27T15:03:20.440 DEBUG:teuthology.orchestra.run.smithi085:> sudo service auditd rotate 2022-11-27T15:03:20.512 INFO:teuthology.orchestra.run.smithi085.stdout:Rotating logs: 2022-11-27T15:03:20.514 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-11-27T15:03:20.514 DEBUG:teuthology.orchestra.run.smithi035:> /usr/sbin/getenforce 2022-11-27T15:03:20.538 INFO:teuthology.orchestra.run.smithi035.stdout:Permissive 2022-11-27T15:03:20.539 DEBUG:teuthology.orchestra.run.smithi085:> /usr/sbin/getenforce 2022-11-27T15:03:20.560 INFO:teuthology.orchestra.run.smithi085.stdout:Permissive 2022-11-27T15:03:20.561 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi035.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi085.front.sepia.ceph.com': 'permissive'} 2022-11-27T15:03:20.561 DEBUG:teuthology.orchestra.run.smithi035:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T15:03:20.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T15:03:20.609 DEBUG:teuthology.orchestra.run.smithi085:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T15:03:20.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T15:03:20.639 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-11-27T15:03:20.639 DEBUG:teuthology.orchestra.run.smithi035:> sudo /usr/sbin/setenforce permissive 2022-11-27T15:03:20.679 DEBUG:teuthology.orchestra.run.smithi085:> sudo /usr/sbin/setenforce permissive 2022-11-27T15:03:20.707 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-11-27T15:03:20.724 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2022-11-27T15:03:20.726 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-11-27T15:03:20.753 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-11-27T15:03:20.755 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi035.front.sepia.ceph.com,smithi085.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-11-27T15:11:34.302 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi035.front.sepia.ceph.com'), Remote(name='ubuntu@smithi085.front.sepia.ceph.com')] 2022-11-27T15:11:34.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi035.front.sepia.ceph.com' 2022-11-27T15:11:34.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T15:11:34.473 DEBUG:teuthology.orchestra.run.smithi035:> true 2022-11-27T15:11:34.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi085.front.sepia.ceph.com' 2022-11-27T15:11:34.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T15:11:34.627 DEBUG:teuthology.orchestra.run.smithi085:> true 2022-11-27T15:11:34.698 INFO:teuthology.run_tasks:Running task clock... 2022-11-27T15:11:34.719 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-11-27T15:11:34.720 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T15:11:34.723 DEBUG:teuthology.orchestra.run.smithi035:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T15:11:34.726 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T15:11:34.726 DEBUG:teuthology.orchestra.run.smithi085:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T15:11:34.761 INFO:teuthology.orchestra.run.smithi035.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T15:11:34.774 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T15:11:34.778 INFO:teuthology.orchestra.run.smithi035.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T15:11:34.788 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T15:11:34.848 INFO:teuthology.orchestra.run.smithi035.stderr:sudo: ntpd: command not found 2022-11-27T15:11:34.861 INFO:teuthology.orchestra.run.smithi035.stdout:506 Cannot talk to daemon 2022-11-27T15:11:34.862 INFO:teuthology.orchestra.run.smithi085.stderr:sudo: ntpd: command not found 2022-11-27T15:11:34.874 INFO:teuthology.orchestra.run.smithi085.stdout:506 Cannot talk to daemon 2022-11-27T15:11:34.878 INFO:teuthology.orchestra.run.smithi035.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T15:11:34.888 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T15:11:34.893 INFO:teuthology.orchestra.run.smithi035.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T15:11:34.902 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T15:11:34.970 INFO:teuthology.orchestra.run.smithi035.stderr:bash: ntpq: command not found 2022-11-27T15:11:34.975 INFO:teuthology.orchestra.run.smithi035.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T15:11:34.975 INFO:teuthology.orchestra.run.smithi035.stdout:=============================================================================== 2022-11-27T15:11:34.975 INFO:teuthology.orchestra.run.smithi035.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T15:11:34.976 INFO:teuthology.orchestra.run.smithi035.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T15:11:34.976 INFO:teuthology.orchestra.run.smithi035.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T15:11:34.976 INFO:teuthology.orchestra.run.smithi035.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T15:11:34.977 INFO:teuthology.orchestra.run.smithi085.stderr:bash: ntpq: command not found 2022-11-27T15:11:34.980 INFO:teuthology.orchestra.run.smithi085.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T15:11:34.980 INFO:teuthology.orchestra.run.smithi085.stdout:=============================================================================== 2022-11-27T15:11:34.980 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T15:11:34.981 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T15:11:34.981 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T15:11:34.981 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T15:11:34.982 INFO:teuthology.run_tasks:Running task install... 2022-11-27T15:11:34.991 DEBUG:teuthology.task.install:project ceph 2022-11-27T15:11:34.991 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}} 2022-11-27T15:11:34.992 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'} 2022-11-27T15:11:34.992 INFO:teuthology.task.install:Using flavor: default 2022-11-27T15:11:34.999 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-11-27T15:11:35.000 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2022-11-27T15:11:35.000 DEBUG:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2022-11-27T15:11:35.060 INFO:teuthology.orchestra.run.smithi035.stdout:RedHatEnterprise 2022-11-27T15:11:35.062 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T15:11:35.063 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2022-11-27T15:11:35.064 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T15:11:35.065 DEBUG:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2022-11-27T15:11:35.107 INFO:teuthology.orchestra.run.smithi085.stdout:RedHatEnterprise 2022-11-27T15:11:35.108 INFO:teuthology.orchestra.run.smithi035.stdout:RedHatEnterprise 2022-11-27T15:11:35.109 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T15:11:35.109 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T15:11:35.110 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2022-11-27T15:11:35.111 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T15:11:35.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T15:11:35.188 INFO:teuthology.orchestra.run.smithi085.stdout:RedHatEnterprise 2022-11-27T15:11:35.190 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T15:11:35.190 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T15:11:35.302 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T15:11:35.302 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T15:11:35.365 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T15:11:35.366 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T15:11:35.439 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T15:11:35.440 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:11:35.440 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T15:11:35.469 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T15:11:35.470 DEBUG:teuthology.orchestra.run.smithi035:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T15:11:35.483 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T15:11:35.483 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:11:35.484 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T15:11:35.512 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T15:11:35.513 DEBUG:teuthology.orchestra.run.smithi085:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T15:11:35.540 DEBUG:teuthology.orchestra.run.smithi035:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T15:11:35.581 DEBUG:teuthology.orchestra.run.smithi085:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T15:11:35.620 DEBUG:teuthology.orchestra.run.smithi035:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T15:11:35.658 DEBUG:teuthology.orchestra.run.smithi085:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T15:11:35.686 INFO:teuthology.orchestra.run.smithi035.stdout:check_obsoletes = 1 2022-11-27T15:11:35.688 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum clean all 2022-11-27T15:11:35.724 INFO:teuthology.orchestra.run.smithi085.stdout:check_obsoletes = 1 2022-11-27T15:11:35.726 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum clean all 2022-11-27T15:11:36.079 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2022-11-27T15:11:36.097 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2022-11-27T15:11:36.297 INFO:teuthology.orchestra.run.smithi035.stdout:54 files removed 2022-11-27T15:11:36.300 INFO:teuthology.orchestra.run.smithi085.stdout:54 files removed 2022-11-27T15:11:36.337 DEBUG:teuthology.orchestra.run.smithi035:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T15:11:36.349 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T15:11:36.680 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2022-11-27T15:11:36.686 INFO:teuthology.orchestra.run.smithi035.stdout:Updating Subscription Management repositories. 2022-11-27T15:11:37.279 INFO:teuthology.orchestra.run.smithi085.stdout:ceph packages for x86_64 281 kB/s | 81 kB 00:00 2022-11-27T15:11:37.320 INFO:teuthology.orchestra.run.smithi035.stdout:ceph packages for x86_64 262 kB/s | 81 kB 00:00 2022-11-27T15:11:37.546 INFO:teuthology.orchestra.run.smithi085.stdout:ceph noarch packages 40 kB/s | 9.4 kB 00:00 2022-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi035.stdout:ceph noarch packages 39 kB/s | 9.4 kB 00:00 2022-11-27T15:11:37.797 INFO:teuthology.orchestra.run.smithi085.stdout:ceph source packages 6.6 kB/s | 1.5 kB 00:00 2022-11-27T15:11:37.872 INFO:teuthology.orchestra.run.smithi035.stdout:ceph source packages 6.0 kB/s | 1.5 kB 00:00 2022-11-27T15:11:40.373 INFO:teuthology.orchestra.run.smithi085.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2022-11-27T15:11:40.581 INFO:teuthology.orchestra.run.smithi035.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2022-11-27T15:11:41.456 INFO:teuthology.orchestra.run.smithi085.stdout:Extra Packages for Enterprise Linux 12 MB/s | 13 MB 00:01 2022-11-27T15:11:42.089 INFO:teuthology.orchestra.run.smithi035.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 13 MB 00:01 2022-11-27T15:11:44.481 INFO:teuthology.orchestra.run.smithi085.stdout:lab-extras 319 kB/s | 24 kB 00:00 2022-11-27T15:11:45.267 INFO:teuthology.orchestra.run.smithi085.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 66 MB/s | 48 MB 00:00 2022-11-27T15:11:45.290 INFO:teuthology.orchestra.run.smithi035.stdout:lab-extras 423 kB/s | 24 kB 00:00 2022-11-27T15:11:46.242 INFO:teuthology.orchestra.run.smithi035.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2022-11-27T15:11:53.343 INFO:teuthology.orchestra.run.smithi085.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 71 MB/s | 53 MB 00:00 2022-11-27T15:11:54.830 INFO:teuthology.orchestra.run.smithi035.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 48 MB 00:00 2022-11-27T15:12:04.533 INFO:teuthology.orchestra.run.smithi085.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T15:12:04.534 INFO:teuthology.orchestra.run.smithi085.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T15:12:04.548 INFO:teuthology.orchestra.run.smithi085.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-11-27T15:12:04.549 INFO:teuthology.orchestra.run.smithi085.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-11-27T15:12:04.551 INFO:teuthology.orchestra.run.smithi085.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-11-27T15:12:04.552 INFO:teuthology.orchestra.run.smithi085.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T15:12:04.554 INFO:teuthology.orchestra.run.smithi085.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T15:12:04.558 INFO:teuthology.orchestra.run.smithi085.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T15:12:04.559 INFO:teuthology.orchestra.run.smithi085.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T15:12:04.690 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2022-11-27T15:12:04.702 INFO:teuthology.orchestra.run.smithi085.stdout:=========================================================================================================================== 2022-11-27T15:12:04.703 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2022-11-27T15:12:04.703 INFO:teuthology.orchestra.run.smithi085.stdout:=========================================================================================================================== 2022-11-27T15:12:04.703 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2022-11-27T15:12:04.704 INFO:teuthology.orchestra.run.smithi085.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-11-27T15:12:04.704 INFO:teuthology.orchestra.run.smithi085.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-11-27T15:12:04.704 INFO:teuthology.orchestra.run.smithi085.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T15:12:04.704 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T15:12:04.704 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T15:12:04.705 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T15:12:04.705 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T15:12:04.705 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T15:12:04.705 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T15:12:04.706 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T15:12:04.706 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T15:12:04.706 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T15:12:04.706 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T15:12:04.707 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T15:12:04.707 INFO:teuthology.orchestra.run.smithi085.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T15:12:04.707 INFO:teuthology.orchestra.run.smithi085.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T15:12:04.707 INFO:teuthology.orchestra.run.smithi085.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T15:12:04.708 INFO:teuthology.orchestra.run.smithi085.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-11-27T15:12:04.708 INFO:teuthology.orchestra.run.smithi085.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-11-27T15:12:04.708 INFO:teuthology.orchestra.run.smithi085.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-11-27T15:12:04.708 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T15:12:04.709 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T15:12:04.709 INFO:teuthology.orchestra.run.smithi085.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T15:12:04.709 INFO:teuthology.orchestra.run.smithi085.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-11-27T15:12:04.709 INFO:teuthology.orchestra.run.smithi085.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-11-27T15:12:04.710 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-11-27T15:12:04.710 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-11-27T15:12:04.710 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-11-27T15:12:04.710 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-11-27T15:12:04.710 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T15:12:04.711 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T15:12:04.711 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T15:12:04.711 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T15:12:04.711 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T15:12:04.712 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T15:12:04.712 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T15:12:04.712 INFO:teuthology.orchestra.run.smithi085.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-11-27T15:12:04.712 INFO:teuthology.orchestra.run.smithi085.stdout:Upgrading: 2022-11-27T15:12:04.713 INFO:teuthology.orchestra.run.smithi085.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-11-27T15:12:04.713 INFO:teuthology.orchestra.run.smithi085.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T15:12:04.713 INFO:teuthology.orchestra.run.smithi085.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T15:12:04.713 INFO:teuthology.orchestra.run.smithi085.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-11-27T15:12:04.714 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-11-27T15:12:04.714 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2022-11-27T15:12:04.714 INFO:teuthology.orchestra.run.smithi085.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-11-27T15:12:04.715 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T15:12:04.715 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T15:12:04.715 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T15:12:04.715 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T15:12:04.716 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T15:12:04.716 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T15:12:04.716 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T15:12:04.716 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T15:12:04.717 INFO:teuthology.orchestra.run.smithi085.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-11-27T15:12:04.717 INFO:teuthology.orchestra.run.smithi085.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T15:12:04.717 INFO:teuthology.orchestra.run.smithi085.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-11-27T15:12:04.717 INFO:teuthology.orchestra.run.smithi085.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-11-27T15:12:04.718 INFO:teuthology.orchestra.run.smithi085.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-11-27T15:12:04.718 INFO:teuthology.orchestra.run.smithi085.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T15:12:04.718 INFO:teuthology.orchestra.run.smithi085.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-11-27T15:12:04.718 INFO:teuthology.orchestra.run.smithi085.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-11-27T15:12:04.719 INFO:teuthology.orchestra.run.smithi085.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T15:12:04.719 INFO:teuthology.orchestra.run.smithi085.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-11-27T15:12:04.719 INFO:teuthology.orchestra.run.smithi085.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-11-27T15:12:04.720 INFO:teuthology.orchestra.run.smithi085.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-11-27T15:12:04.720 INFO:teuthology.orchestra.run.smithi085.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T15:12:04.720 INFO:teuthology.orchestra.run.smithi085.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-11-27T15:12:04.720 INFO:teuthology.orchestra.run.smithi085.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T15:12:04.721 INFO:teuthology.orchestra.run.smithi085.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-11-27T15:12:04.721 INFO:teuthology.orchestra.run.smithi085.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-11-27T15:12:04.721 INFO:teuthology.orchestra.run.smithi085.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T15:12:04.721 INFO:teuthology.orchestra.run.smithi085.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-11-27T15:12:04.721 INFO:teuthology.orchestra.run.smithi085.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-11-27T15:12:04.722 INFO:teuthology.orchestra.run.smithi085.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-11-27T15:12:04.722 INFO:teuthology.orchestra.run.smithi085.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-11-27T15:12:04.722 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-11-27T15:12:04.722 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-11-27T15:12:04.723 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-11-27T15:12:04.723 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-11-27T15:12:04.723 INFO:teuthology.orchestra.run.smithi085.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T15:12:04.724 INFO:teuthology.orchestra.run.smithi085.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-11-27T15:12:04.724 INFO:teuthology.orchestra.run.smithi085.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T15:12:04.724 INFO:teuthology.orchestra.run.smithi085.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T15:12:04.724 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T15:12:04.725 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T15:12:04.725 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T15:12:04.725 INFO:teuthology.orchestra.run.smithi085.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T15:12:04.725 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-11-27T15:12:04.725 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T15:12:04.726 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T15:12:04.726 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-11-27T15:12:04.726 INFO:teuthology.orchestra.run.smithi085.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T15:12:04.726 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T15:12:04.727 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T15:12:04.727 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-11-27T15:12:04.727 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-11-27T15:12:04.727 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-11-27T15:12:04.728 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T15:12:04.728 INFO:teuthology.orchestra.run.smithi085.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T15:12:04.728 INFO:teuthology.orchestra.run.smithi085.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T15:12:04.728 INFO:teuthology.orchestra.run.smithi085.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-11-27T15:12:04.728 INFO:teuthology.orchestra.run.smithi085.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-11-27T15:12:04.729 INFO:teuthology.orchestra.run.smithi085.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T15:12:04.729 INFO:teuthology.orchestra.run.smithi085.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T15:12:04.729 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-11-27T15:12:04.729 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-11-27T15:12:04.730 INFO:teuthology.orchestra.run.smithi085.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-11-27T15:12:04.730 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T15:12:04.730 INFO:teuthology.orchestra.run.smithi085.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T15:12:04.730 INFO:teuthology.orchestra.run.smithi085.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-11-27T15:12:04.731 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-11-27T15:12:04.731 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-11-27T15:12:04.731 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-11-27T15:12:04.731 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-11-27T15:12:04.732 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-11-27T15:12:04.732 INFO:teuthology.orchestra.run.smithi085.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T15:12:04.732 INFO:teuthology.orchestra.run.smithi085.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T15:12:04.732 INFO:teuthology.orchestra.run.smithi085.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T15:12:04.733 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T15:12:04.733 INFO:teuthology.orchestra.run.smithi085.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-11-27T15:12:04.733 INFO:teuthology.orchestra.run.smithi085.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T15:12:04.733 INFO:teuthology.orchestra.run.smithi085.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T15:12:04.734 INFO:teuthology.orchestra.run.smithi085.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T15:12:04.734 INFO:teuthology.orchestra.run.smithi085.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T15:12:04.734 INFO:teuthology.orchestra.run.smithi085.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T15:12:04.734 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T15:12:04.734 INFO:teuthology.orchestra.run.smithi085.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T15:12:04.735 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T15:12:04.735 INFO:teuthology.orchestra.run.smithi085.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-11-27T15:12:04.735 INFO:teuthology.orchestra.run.smithi085.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T15:12:04.735 INFO:teuthology.orchestra.run.smithi085.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-11-27T15:12:04.736 INFO:teuthology.orchestra.run.smithi085.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T15:12:04.736 INFO:teuthology.orchestra.run.smithi085.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-11-27T15:12:04.736 INFO:teuthology.orchestra.run.smithi085.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-11-27T15:12:04.736 INFO:teuthology.orchestra.run.smithi085.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-11-27T15:12:04.737 INFO:teuthology.orchestra.run.smithi085.stdout:Enabling module streams: 2022-11-27T15:12:04.737 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql 10 2022-11-27T15:12:04.737 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:12:04.737 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2022-11-27T15:12:04.738 INFO:teuthology.orchestra.run.smithi085.stdout:=========================================================================================================================== 2022-11-27T15:12:04.738 INFO:teuthology.orchestra.run.smithi085.stdout:Install 128 Packages 2022-11-27T15:12:04.738 INFO:teuthology.orchestra.run.smithi085.stdout:Upgrade 5 Packages 2022-11-27T15:12:04.738 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:12:04.739 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 219 M 2022-11-27T15:12:04.739 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2022-11-27T15:12:04.842 INFO:teuthology.orchestra.run.smithi085.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 52 kB/s | 6.4 kB 00:00 2022-11-27T15:12:05.067 INFO:teuthology.orchestra.run.smithi085.stdout:(2/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 3.7 MB/s | 845 kB 00:00 2022-11-27T15:12:05.109 INFO:teuthology.orchestra.run.smithi085.stdout:(3/133): ceph-immutable-object-cache-18.0.0-594 3.9 MB/s | 165 kB 00:00 2022-11-27T15:12:05.418 INFO:teuthology.orchestra.run.smithi085.stdout:(4/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 7.4 MB/s | 5.2 MB 00:00 2022-11-27T15:12:05.468 INFO:teuthology.orchestra.run.smithi085.stdout:(5/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 6.3 MB/s | 2.3 MB 00:00 2022-11-27T15:12:05.602 INFO:teuthology.orchestra.run.smithi085.stdout:(6/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 8.3 MB/s | 1.5 MB 00:00 2022-11-27T15:12:06.060 INFO:teuthology.orchestra.run.smithi085.stdout:(7/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 6.5 MB/s | 3.9 MB 00:00 2022-11-27T15:12:06.585 INFO:teuthology.orchestra.run.smithi035.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T15:12:06.586 INFO:teuthology.orchestra.run.smithi035.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T15:12:06.609 INFO:teuthology.orchestra.run.smithi035.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-11-27T15:12:06.609 INFO:teuthology.orchestra.run.smithi035.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-11-27T15:12:06.610 INFO:teuthology.orchestra.run.smithi035.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-11-27T15:12:06.610 INFO:teuthology.orchestra.run.smithi035.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T15:12:06.610 INFO:teuthology.orchestra.run.smithi035.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T15:12:06.611 INFO:teuthology.orchestra.run.smithi035.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T15:12:06.613 INFO:teuthology.orchestra.run.smithi035.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T15:12:06.757 INFO:teuthology.orchestra.run.smithi035.stdout:Dependencies resolved. 2022-11-27T15:12:06.771 INFO:teuthology.orchestra.run.smithi035.stdout:=========================================================================================================================== 2022-11-27T15:12:06.772 INFO:teuthology.orchestra.run.smithi035.stdout: Package Arch Version Repository Size 2022-11-27T15:12:06.772 INFO:teuthology.orchestra.run.smithi035.stdout:=========================================================================================================================== 2022-11-27T15:12:06.772 INFO:teuthology.orchestra.run.smithi035.stdout:Installing: 2022-11-27T15:12:06.773 INFO:teuthology.orchestra.run.smithi035.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-11-27T15:12:06.773 INFO:teuthology.orchestra.run.smithi035.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-11-27T15:12:06.773 INFO:teuthology.orchestra.run.smithi035.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T15:12:06.773 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T15:12:06.774 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T15:12:06.774 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T15:12:06.774 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T15:12:06.774 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T15:12:06.774 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T15:12:06.775 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T15:12:06.775 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T15:12:06.775 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T15:12:06.775 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T15:12:06.776 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T15:12:06.776 INFO:teuthology.orchestra.run.smithi035.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T15:12:06.776 INFO:teuthology.orchestra.run.smithi035.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T15:12:06.776 INFO:teuthology.orchestra.run.smithi035.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T15:12:06.776 INFO:teuthology.orchestra.run.smithi035.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-11-27T15:12:06.777 INFO:teuthology.orchestra.run.smithi035.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-11-27T15:12:06.777 INFO:teuthology.orchestra.run.smithi035.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-11-27T15:12:06.777 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T15:12:06.777 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T15:12:06.778 INFO:teuthology.orchestra.run.smithi035.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T15:12:06.778 INFO:teuthology.orchestra.run.smithi035.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-11-27T15:12:06.778 INFO:teuthology.orchestra.run.smithi035.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-11-27T15:12:06.778 INFO:teuthology.orchestra.run.smithi035.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-11-27T15:12:06.779 INFO:teuthology.orchestra.run.smithi035.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-11-27T15:12:06.779 INFO:teuthology.orchestra.run.smithi035.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-11-27T15:12:06.779 INFO:teuthology.orchestra.run.smithi035.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-11-27T15:12:06.779 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T15:12:06.780 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T15:12:06.780 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T15:12:06.780 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T15:12:06.780 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T15:12:06.781 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T15:12:06.781 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T15:12:06.781 INFO:teuthology.orchestra.run.smithi035.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-11-27T15:12:06.781 INFO:teuthology.orchestra.run.smithi035.stdout:Upgrading: 2022-11-27T15:12:06.781 INFO:teuthology.orchestra.run.smithi035.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-11-27T15:12:06.782 INFO:teuthology.orchestra.run.smithi035.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T15:12:06.782 INFO:teuthology.orchestra.run.smithi035.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T15:12:06.782 INFO:teuthology.orchestra.run.smithi035.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-11-27T15:12:06.783 INFO:teuthology.orchestra.run.smithi035.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-11-27T15:12:06.783 INFO:teuthology.orchestra.run.smithi035.stdout:Installing dependencies: 2022-11-27T15:12:06.783 INFO:teuthology.orchestra.run.smithi035.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-11-27T15:12:06.783 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T15:12:06.784 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T15:12:06.784 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T15:12:06.784 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T15:12:06.784 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T15:12:06.785 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T15:12:06.785 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T15:12:06.785 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T15:12:06.785 INFO:teuthology.orchestra.run.smithi035.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-11-27T15:12:06.785 INFO:teuthology.orchestra.run.smithi035.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T15:12:06.786 INFO:teuthology.orchestra.run.smithi035.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-11-27T15:12:06.786 INFO:teuthology.orchestra.run.smithi035.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-11-27T15:12:06.786 INFO:teuthology.orchestra.run.smithi035.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-11-27T15:12:06.786 INFO:teuthology.orchestra.run.smithi035.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T15:12:06.787 INFO:teuthology.orchestra.run.smithi035.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-11-27T15:12:06.787 INFO:teuthology.orchestra.run.smithi035.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-11-27T15:12:06.787 INFO:teuthology.orchestra.run.smithi035.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T15:12:06.788 INFO:teuthology.orchestra.run.smithi035.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-11-27T15:12:06.788 INFO:teuthology.orchestra.run.smithi035.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-11-27T15:12:06.788 INFO:teuthology.orchestra.run.smithi035.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-11-27T15:12:06.788 INFO:teuthology.orchestra.run.smithi035.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T15:12:06.788 INFO:teuthology.orchestra.run.smithi035.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-11-27T15:12:06.789 INFO:teuthology.orchestra.run.smithi035.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T15:12:06.789 INFO:teuthology.orchestra.run.smithi035.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-11-27T15:12:06.789 INFO:teuthology.orchestra.run.smithi035.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-11-27T15:12:06.789 INFO:teuthology.orchestra.run.smithi035.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T15:12:06.790 INFO:teuthology.orchestra.run.smithi035.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-11-27T15:12:06.790 INFO:teuthology.orchestra.run.smithi035.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-11-27T15:12:06.790 INFO:teuthology.orchestra.run.smithi035.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-11-27T15:12:06.790 INFO:teuthology.orchestra.run.smithi035.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-11-27T15:12:06.790 INFO:teuthology.orchestra.run.smithi035.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-11-27T15:12:06.791 INFO:teuthology.orchestra.run.smithi035.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-11-27T15:12:06.791 INFO:teuthology.orchestra.run.smithi035.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-11-27T15:12:06.791 INFO:teuthology.orchestra.run.smithi035.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-11-27T15:12:06.792 INFO:teuthology.orchestra.run.smithi035.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T15:12:06.792 INFO:teuthology.orchestra.run.smithi035.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-11-27T15:12:06.792 INFO:teuthology.orchestra.run.smithi035.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T15:12:06.792 INFO:teuthology.orchestra.run.smithi035.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T15:12:06.793 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T15:12:06.793 INFO:teuthology.orchestra.run.smithi035.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T15:12:06.793 INFO:teuthology.orchestra.run.smithi035.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T15:12:06.793 INFO:teuthology.orchestra.run.smithi035.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T15:12:06.794 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-11-27T15:12:06.794 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T15:12:06.794 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T15:12:06.794 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-11-27T15:12:06.794 INFO:teuthology.orchestra.run.smithi035.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T15:12:06.795 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T15:12:06.795 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T15:12:06.795 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-11-27T15:12:06.795 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-11-27T15:12:06.796 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-11-27T15:12:06.796 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T15:12:06.796 INFO:teuthology.orchestra.run.smithi035.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T15:12:06.796 INFO:teuthology.orchestra.run.smithi035.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T15:12:06.797 INFO:teuthology.orchestra.run.smithi035.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-11-27T15:12:06.797 INFO:teuthology.orchestra.run.smithi035.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-11-27T15:12:06.797 INFO:teuthology.orchestra.run.smithi035.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T15:12:06.797 INFO:teuthology.orchestra.run.smithi035.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T15:12:06.798 INFO:teuthology.orchestra.run.smithi035.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-11-27T15:12:06.798 INFO:teuthology.orchestra.run.smithi035.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-11-27T15:12:06.798 INFO:teuthology.orchestra.run.smithi035.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-11-27T15:12:06.798 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T15:12:06.798 INFO:teuthology.orchestra.run.smithi035.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T15:12:06.799 INFO:teuthology.orchestra.run.smithi035.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-11-27T15:12:06.799 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-11-27T15:12:06.799 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-11-27T15:12:06.799 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-11-27T15:12:06.800 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-11-27T15:12:06.800 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-11-27T15:12:06.800 INFO:teuthology.orchestra.run.smithi035.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T15:12:06.800 INFO:teuthology.orchestra.run.smithi035.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T15:12:06.801 INFO:teuthology.orchestra.run.smithi035.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T15:12:06.801 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T15:12:06.801 INFO:teuthology.orchestra.run.smithi035.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-11-27T15:12:06.802 INFO:teuthology.orchestra.run.smithi035.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T15:12:06.802 INFO:teuthology.orchestra.run.smithi035.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T15:12:06.802 INFO:teuthology.orchestra.run.smithi035.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T15:12:06.802 INFO:teuthology.orchestra.run.smithi035.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T15:12:06.802 INFO:teuthology.orchestra.run.smithi035.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T15:12:06.803 INFO:teuthology.orchestra.run.smithi035.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T15:12:06.803 INFO:teuthology.orchestra.run.smithi035.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T15:12:06.803 INFO:teuthology.orchestra.run.smithi035.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T15:12:06.804 INFO:teuthology.orchestra.run.smithi035.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-11-27T15:12:06.804 INFO:teuthology.orchestra.run.smithi035.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T15:12:06.804 INFO:teuthology.orchestra.run.smithi035.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-11-27T15:12:06.804 INFO:teuthology.orchestra.run.smithi035.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T15:12:06.805 INFO:teuthology.orchestra.run.smithi035.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-11-27T15:12:06.805 INFO:teuthology.orchestra.run.smithi035.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-11-27T15:12:06.805 INFO:teuthology.orchestra.run.smithi035.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-11-27T15:12:06.805 INFO:teuthology.orchestra.run.smithi035.stdout:Enabling module streams: 2022-11-27T15:12:06.805 INFO:teuthology.orchestra.run.smithi035.stdout: postgresql 10 2022-11-27T15:12:06.806 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:12:06.806 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction Summary 2022-11-27T15:12:06.806 INFO:teuthology.orchestra.run.smithi035.stdout:=========================================================================================================================== 2022-11-27T15:12:06.806 INFO:teuthology.orchestra.run.smithi035.stdout:Install 128 Packages 2022-11-27T15:12:06.807 INFO:teuthology.orchestra.run.smithi035.stdout:Upgrade 5 Packages 2022-11-27T15:12:06.807 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:12:06.808 INFO:teuthology.orchestra.run.smithi035.stdout:Total download size: 219 M 2022-11-27T15:12:06.808 INFO:teuthology.orchestra.run.smithi035.stdout:Downloading Packages: 2022-11-27T15:12:07.101 INFO:teuthology.orchestra.run.smithi035.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 21 kB/s | 6.4 kB 00:00 2022-11-27T15:12:07.252 INFO:teuthology.orchestra.run.smithi085.stdout:(8/133): ceph-common-18.0.0-594.g697b0d99.el8.x 9.6 MB/s | 24 MB 00:02 2022-11-27T15:12:07.402 INFO:teuthology.orchestra.run.smithi035.stdout:(2/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 2.8 MB/s | 845 kB 00:00 2022-11-27T15:12:07.436 INFO:teuthology.orchestra.run.smithi085.stdout:(9/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86_ 9.2 MB/s | 17 MB 00:01 2022-11-27T15:12:07.453 INFO:teuthology.orchestra.run.smithi085.stdout:(10/133): ceph-selinux-18.0.0-594.g697b0d99.el8 121 kB/s | 24 kB 00:00 2022-11-27T15:12:07.485 INFO:teuthology.orchestra.run.smithi035.stdout:(3/133): ceph-immutable-object-cache-18.0.0-594 1.9 MB/s | 165 kB 00:00 2022-11-27T15:12:07.528 INFO:teuthology.orchestra.run.smithi085.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 3.3 MB/s | 257 kB 00:00 2022-11-27T15:12:07.611 INFO:teuthology.orchestra.run.smithi085.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 366 kB/s | 30 kB 00:00 2022-11-27T15:12:08.227 INFO:teuthology.orchestra.run.smithi035.stdout:(4/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 3.6 MB/s | 5.2 MB 00:01 2022-11-27T15:12:08.445 INFO:teuthology.orchestra.run.smithi085.stdout:(13/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 921 kB/s | 767 kB 00:00 2022-11-27T15:12:08.452 INFO:teuthology.orchestra.run.smithi035.stdout:(5/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 2.3 MB/s | 2.3 MB 00:00 2022-11-27T15:12:08.754 INFO:teuthology.orchestra.run.smithi085.stdout:(14/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 6.8 MB/s | 18 MB 00:02 2022-11-27T15:12:08.779 INFO:teuthology.orchestra.run.smithi085.stdout:(15/133): libcephsqlite-18.0.0-594.g697b0d99.el 411 kB/s | 137 kB 00:00 2022-11-27T15:12:08.811 INFO:teuthology.orchestra.run.smithi035.stdout:(6/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 2.6 MB/s | 1.5 MB 00:00 2022-11-27T15:12:08.838 INFO:teuthology.orchestra.run.smithi085.stdout:(16/133): librados-devel-18.0.0-594.g697b0d99.e 1.6 MB/s | 139 kB 00:00 2022-11-27T15:12:09.297 INFO:teuthology.orchestra.run.smithi085.stdout:(17/133): libradosstriper1-18.0.0-594.g697b0d99 918 kB/s | 474 kB 00:00 2022-11-27T15:12:09.372 INFO:teuthology.orchestra.run.smithi085.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 596 kB/s | 45 kB 00:00 2022-11-27T15:12:09.541 INFO:teuthology.orchestra.run.smithi085.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 655 kB/s | 109 kB 00:00 2022-11-27T15:12:09.705 INFO:teuthology.orchestra.run.smithi085.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 1.1 MB/s | 190 kB 00:00 2022-11-27T15:12:09.939 INFO:teuthology.orchestra.run.smithi085.stdout:(21/133): python3-rados-18.0.0-594.g697b0d99.el 1.6 MB/s | 371 kB 00:00 2022-11-27T15:12:10.028 INFO:teuthology.orchestra.run.smithi035.stdout:(7/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 2.5 MB/s | 3.9 MB 00:01 2022-11-27T15:12:10.164 INFO:teuthology.orchestra.run.smithi085.stdout:(22/133): python3-rbd-18.0.0-594.g697b0d99.el8. 1.6 MB/s | 357 kB 00:00 2022-11-27T15:12:10.256 INFO:teuthology.orchestra.run.smithi085.stdout:(23/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 4.6 MB/s | 6.5 MB 00:01 2022-11-27T15:12:10.282 INFO:teuthology.orchestra.run.smithi085.stdout:(24/133): python3-rgw-18.0.0-594.g697b0d99.el8. 912 kB/s | 107 kB 00:00 2022-11-27T15:12:10.340 INFO:teuthology.orchestra.run.smithi085.stdout:(25/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 1.1 MB/s | 94 kB 00:00 2022-11-27T15:12:10.440 INFO:teuthology.orchestra.run.smithi085.stdout:(26/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 1.7 MB/s | 175 kB 00:00 2022-11-27T15:12:10.532 INFO:teuthology.orchestra.run.smithi085.stdout:(27/133): ceph-grafana-dashboards-18.0.0-594.g6 249 kB/s | 23 kB 00:00 2022-11-27T15:12:10.599 INFO:teuthology.orchestra.run.smithi085.stdout:(28/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 1.7 MB/s | 118 kB 00:00 2022-11-27T15:12:10.725 INFO:teuthology.orchestra.run.smithi035.stdout:(8/133): ceph-common-18.0.0-594.g697b0d99.el8.x 6.2 MB/s | 24 MB 00:03 2022-11-27T15:12:10.811 INFO:teuthology.orchestra.run.smithi035.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 281 kB/s | 24 kB 00:00 2022-11-27T15:12:10.908 INFO:teuthology.orchestra.run.smithi085.stdout:(29/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 5.4 MB/s | 1.7 MB 00:00 2022-11-27T15:12:11.333 INFO:teuthology.orchestra.run.smithi085.stdout:(30/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 3.3 MB/s | 3.5 MB 00:01 2022-11-27T15:12:11.442 INFO:teuthology.orchestra.run.smithi085.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 2.1 MB/s | 234 kB 00:00 2022-11-27T15:12:11.542 INFO:teuthology.orchestra.run.smithi085.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 469 kB/s | 47 kB 00:00 2022-11-27T15:12:11.642 INFO:teuthology.orchestra.run.smithi085.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 144 kB/s | 14 kB 00:00 2022-11-27T15:12:11.751 INFO:teuthology.orchestra.run.smithi085.stdout:(34/133): ceph-volume-18.0.0-594.g697b0d99.el8. 2.4 MB/s | 263 kB 00:00 2022-11-27T15:12:11.843 INFO:teuthology.orchestra.run.smithi085.stdout:(35/133): cephadm-18.0.0-594.g697b0d99.el8.noar 889 kB/s | 81 kB 00:00 2022-11-27T15:12:11.943 INFO:teuthology.orchestra.run.smithi085.stdout:(36/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 154 kB/s | 15 kB 00:00 2022-11-27T15:12:12.144 INFO:teuthology.orchestra.run.smithi085.stdout:(37/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.5 MB/s | 306 kB 00:00 2022-11-27T15:12:12.235 INFO:teuthology.orchestra.run.smithi085.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 641 kB/s | 59 kB 00:00 2022-11-27T15:12:12.302 INFO:teuthology.orchestra.run.smithi085.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2022-11-27T15:12:12.528 INFO:teuthology.orchestra.run.smithi085.stdout:(40/133): ceph-mgr-diskprediction-local-18.0.0- 4.6 MB/s | 7.4 MB 00:01 2022-11-27T15:12:12.553 INFO:teuthology.orchestra.run.smithi085.stdout:(41/133): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2022-11-27T15:12:12.628 INFO:teuthology.orchestra.run.smithi085.stdout:(42/133): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2022-11-27T15:12:12.687 INFO:teuthology.orchestra.run.smithi085.stdout:(43/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 274 kB/s | 44 kB 00:00 2022-11-27T15:12:12.704 INFO:teuthology.orchestra.run.smithi085.stdout:(44/133): python3-cachetools-3.1.1-4.el8.noarch 437 kB/s | 33 kB 00:00 2022-11-27T15:12:12.763 INFO:teuthology.orchestra.run.smithi085.stdout:(45/133): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2022-11-27T15:12:12.796 INFO:teuthology.orchestra.run.smithi085.stdout:(46/133): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2022-11-27T15:12:12.896 INFO:teuthology.orchestra.run.smithi085.stdout:(47/133): python3-google-auth-1.1.1-10.el8.noar 814 kB/s | 81 kB 00:00 2022-11-27T15:12:12.922 INFO:teuthology.orchestra.run.smithi085.stdout:(48/133): python3-cherrypy-18.4.0-1.el8.noarch. 2.4 MB/s | 384 kB 00:00 2022-11-27T15:12:12.972 INFO:teuthology.orchestra.run.smithi085.stdout:(49/133): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2022-11-27T15:12:12.989 INFO:teuthology.orchestra.run.smithi085.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2022-11-27T15:12:13.048 INFO:teuthology.orchestra.run.smithi085.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 833 kB/s | 49 kB 00:00 2022-11-27T15:12:13.123 INFO:teuthology.orchestra.run.smithi085.stdout:(52/133): python3-more-itertools-7.2.0-3.el8.no 784 kB/s | 59 kB 00:00 2022-11-27T15:12:13.170 INFO:teuthology.orchestra.run.smithi035.stdout:(10/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86 3.9 MB/s | 17 MB 00:04 2022-11-27T15:12:13.253 INFO:teuthology.orchestra.run.smithi035.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 3.0 MB/s | 257 kB 00:00 2022-11-27T15:12:13.320 INFO:teuthology.orchestra.run.smithi035.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 458 kB/s | 30 kB 00:00 2022-11-27T15:12:13.479 INFO:teuthology.orchestra.run.smithi035.stdout:(13/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 4.7 MB/s | 767 kB 00:00 2022-11-27T15:12:13.554 INFO:teuthology.orchestra.run.smithi035.stdout:(14/133): libcephsqlite-18.0.0-594.g697b0d99.el 1.8 MB/s | 137 kB 00:00 2022-11-27T15:12:13.573 INFO:teuthology.orchestra.run.smithi085.stdout:(53/133): ceph-test-18.0.0-594.g697b0d99.el8.x8 8.9 MB/s | 55 MB 00:06 2022-11-27T15:12:13.590 INFO:teuthology.orchestra.run.smithi085.stdout:(54/133): python3-natsort-7.1.1-2.el8.noarch.rp 129 kB/s | 60 kB 00:00 2022-11-27T15:12:13.624 INFO:teuthology.orchestra.run.smithi085.stdout:(55/133): python3-kubernetes-11.0.0-6.el8.noarc 1.8 MB/s | 1.2 MB 00:00 2022-11-27T15:12:13.638 INFO:teuthology.orchestra.run.smithi035.stdout:(15/133): librados-devel-18.0.0-594.g697b0d99.e 1.6 MB/s | 139 kB 00:00 2022-11-27T15:12:13.641 INFO:teuthology.orchestra.run.smithi085.stdout:(56/133): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2022-11-27T15:12:13.691 INFO:teuthology.orchestra.run.smithi085.stdout:(57/133): python3-repoze-lru-0.7-6.el8.noarch.r 503 kB/s | 34 kB 00:00 2022-11-27T15:12:13.716 INFO:teuthology.orchestra.run.smithi085.stdout:(58/133): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2022-11-27T15:12:13.758 INFO:teuthology.orchestra.run.smithi035.stdout:(16/133): libradosstriper1-18.0.0-594.g697b0d99 4.0 MB/s | 474 kB 00:00 2022-11-27T15:12:13.775 INFO:teuthology.orchestra.run.smithi085.stdout:(59/133): python3-rsa-4.8-1.el8.noarch.rpm 736 kB/s | 61 kB 00:00 2022-11-27T15:12:13.800 INFO:teuthology.orchestra.run.smithi085.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 224 kB/s | 19 kB 00:00 2022-11-27T15:12:13.826 INFO:teuthology.orchestra.run.smithi085.stdout:(61/133): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2022-11-27T15:12:13.851 INFO:teuthology.orchestra.run.smithi085.stdout:(62/133): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2022-11-27T15:12:13.876 INFO:teuthology.orchestra.run.smithi085.stdout:(63/133): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2022-11-27T15:12:13.905 INFO:teuthology.orchestra.run.smithi085.stdout:(64/133): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2022-11-27T15:12:13.943 INFO:teuthology.orchestra.run.smithi085.stdout:(65/133): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2022-11-27T15:12:13.968 INFO:teuthology.orchestra.run.smithi085.stdout:(66/133): python3-websocket-client-0.56.0-5.el8 916 kB/s | 61 kB 00:00 2022-11-27T15:12:14.027 INFO:teuthology.orchestra.run.smithi085.stdout:(67/133): python3-waitress-1.2.1-2.el8.1.noarch 1.3 MB/s | 241 kB 00:00 2022-11-27T15:12:14.052 INFO:teuthology.orchestra.run.smithi085.stdout:(68/133): python3-zc-lockfile-2.0-2.el8.noarch. 274 kB/s | 23 kB 00:00 2022-11-27T15:12:14.077 INFO:teuthology.orchestra.run.smithi085.stdout:(69/133): python3-webtest-2.0.33-1.el8.noarch.r 637 kB/s | 85 kB 00:00 2022-11-27T15:12:14.103 INFO:teuthology.orchestra.run.smithi085.stdout:(70/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2022-11-27T15:12:14.170 INFO:teuthology.orchestra.run.smithi085.stdout:(71/133): postgresql-contrib-10.21-2.module+el8 8.6 MB/s | 808 kB 00:00 2022-11-27T15:12:14.204 INFO:teuthology.orchestra.run.smithi085.stdout:(72/133): bison-3.0.4-10.el8.x86_64.rpm 6.8 MB/s | 688 kB 00:00 2022-11-27T15:12:14.262 INFO:teuthology.orchestra.run.smithi085.stdout:(73/133): postgresql-10.21-2.module+el8.6.0+153 16 MB/s | 1.5 MB 00:00 2022-11-27T15:12:14.338 INFO:teuthology.orchestra.run.smithi085.stdout:(74/133): python3-numpy-1.14.3-10.el8.x86_64.rp 28 MB/s | 3.7 MB 00:00 2022-11-27T15:12:14.380 INFO:teuthology.orchestra.run.smithi085.stdout:(75/133): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2022-11-27T15:12:14.422 INFO:teuthology.orchestra.run.smithi085.stdout:(76/133): thrift-0.13.0-2.el8.x86_64.rpm 4.4 MB/s | 1.7 MB 00:00 2022-11-27T15:12:14.455 INFO:teuthology.orchestra.run.smithi085.stdout:(77/133): python3-markupsafe-0.23-19.el8.x86_64 516 kB/s | 39 kB 00:00 2022-11-27T15:12:14.522 INFO:teuthology.orchestra.run.smithi085.stdout:(78/133): postgresql-server-10.21-2.module+el8. 20 MB/s | 5.1 MB 00:00 2022-11-27T15:12:14.547 INFO:teuthology.orchestra.run.smithi085.stdout:(79/133): python3-prettytable-0.7.2-14.el8.noar 354 kB/s | 44 kB 00:00 2022-11-27T15:12:14.589 INFO:teuthology.orchestra.run.smithi085.stdout:(80/133): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2022-11-27T15:12:14.623 INFO:teuthology.orchestra.run.smithi085.stdout:(81/133): python3-jinja2-2.10.1-3.el8.noarch.rp 5.2 MB/s | 538 kB 00:00 2022-11-27T15:12:14.648 INFO:teuthology.orchestra.run.smithi085.stdout:(82/133): libtool-2.4.6-25.el8.x86_64.rpm 7.0 MB/s | 709 kB 00:00 2022-11-27T15:12:14.673 INFO:teuthology.orchestra.run.smithi085.stdout:(83/133): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.6 MB/s | 225 kB 00:00 2022-11-27T15:12:14.715 INFO:teuthology.orchestra.run.smithi085.stdout:(84/133): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2022-11-27T15:12:14.766 INFO:teuthology.orchestra.run.smithi085.stdout:(85/133): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2022-11-27T15:12:14.772 INFO:teuthology.orchestra.run.smithi035.stdout:(17/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 6.4 MB/s | 6.5 MB 00:01 2022-11-27T15:12:14.841 INFO:teuthology.orchestra.run.smithi085.stdout:(86/133): openblas-threads-0.3.15-3.el8.x86_64. 22 MB/s | 4.9 MB 00:00 2022-11-27T15:12:14.864 INFO:teuthology.orchestra.run.smithi035.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 487 kB/s | 45 kB 00:00 2022-11-27T15:12:14.866 INFO:teuthology.orchestra.run.smithi085.stdout:(87/133): socat-1.7.4.1-1.el8.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2022-11-27T15:12:14.939 INFO:teuthology.orchestra.run.smithi035.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 1.4 MB/s | 109 kB 00:00 2022-11-27T15:12:14.941 INFO:teuthology.orchestra.run.smithi085.stdout:(88/133): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2022-11-27T15:12:14.975 INFO:teuthology.orchestra.run.smithi085.stdout:(89/133): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2022-11-27T15:12:14.998 INFO:teuthology.orchestra.run.smithi035.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 3.2 MB/s | 190 kB 00:00 2022-11-27T15:12:15.009 INFO:teuthology.orchestra.run.smithi085.stdout:(90/133): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2022-11-27T15:12:15.059 INFO:teuthology.orchestra.run.smithi085.stdout:(91/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-11-27T15:12:15.106 INFO:teuthology.orchestra.run.smithi035.stdout:(21/133): python3-rados-18.0.0-594.g697b0d99.el 3.4 MB/s | 371 kB 00:00 2022-11-27T15:12:15.126 INFO:teuthology.orchestra.run.smithi085.stdout:(92/133): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2022-11-27T15:12:15.167 INFO:teuthology.orchestra.run.smithi085.stdout:(93/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.0 MB/s | 126 kB 00:00 2022-11-27T15:12:15.185 INFO:teuthology.orchestra.run.smithi085.stdout:(94/133): python3-pytz-2017.2-9.el8.noarch.rpm 432 kB/s | 54 kB 00:00 2022-11-27T15:12:15.218 INFO:teuthology.orchestra.run.smithi085.stdout:(95/133): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2022-11-27T15:12:15.235 INFO:teuthology.orchestra.run.smithi085.stdout:(96/133): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2022-11-27T15:12:15.277 INFO:teuthology.orchestra.run.smithi085.stdout:(97/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2022-11-27T15:12:15.302 INFO:teuthology.orchestra.run.smithi085.stdout:(98/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2022-11-27T15:12:15.336 INFO:teuthology.orchestra.run.smithi085.stdout:(99/133): python3-jsonpatch-1.21-2.el8.noarch.r 454 kB/s | 27 kB 00:00 2022-11-27T15:12:15.361 INFO:teuthology.orchestra.run.smithi085.stdout:(100/133): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2022-11-27T15:12:15.504 INFO:teuthology.orchestra.run.smithi085.stdout:(101/133): python3-scipy-1.0.0-21.module+el8.5. 26 MB/s | 14 MB 00:00 2022-11-27T15:12:15.529 INFO:teuthology.orchestra.run.smithi085.stdout:(102/133): python3-pyasn1-modules-0.3.7-6.el8.n 573 kB/s | 110 kB 00:00 2022-11-27T15:12:15.571 INFO:teuthology.orchestra.run.smithi085.stdout:(103/133): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2022-11-27T15:12:15.596 INFO:teuthology.orchestra.run.smithi085.stdout:(104/133): fuse-2.9.7-15.el8.x86_64.rpm 897 kB/s | 83 kB 00:00 2022-11-27T15:12:15.621 INFO:teuthology.orchestra.run.smithi085.stdout:(105/133): libsepol-devel-2.9-3.el8.x86_64.rpm 952 kB/s | 87 kB 00:00 2022-11-27T15:12:15.640 INFO:teuthology.orchestra.run.smithi035.stdout:(22/133): ceph-test-18.0.0-594.g697b0d99.el8.x8 11 MB/s | 55 MB 00:04 2022-11-27T15:12:15.663 INFO:teuthology.orchestra.run.smithi085.stdout:(106/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.4 MB/s | 220 kB 00:00 2022-11-27T15:12:15.665 INFO:teuthology.orchestra.run.smithi035.stdout:(23/133): python3-rbd-18.0.0-594.g697b0d99.el8. 639 kB/s | 357 kB 00:00 2022-11-27T15:12:15.688 INFO:teuthology.orchestra.run.smithi085.stdout:(107/133): libverto-devel-0.3.0-5.el8.x86_64.rp 271 kB/s | 18 kB 00:00 2022-11-27T15:12:15.690 INFO:teuthology.orchestra.run.smithi035.stdout:(24/133): python3-rgw-18.0.0-594.g697b0d99.el8. 2.1 MB/s | 107 kB 00:00 2022-11-27T15:12:15.714 INFO:teuthology.orchestra.run.smithi085.stdout:(108/133): libselinux-devel-2.9-5.el8.x86_64.rp 1.7 MB/s | 200 kB 00:00 2022-11-27T15:12:15.716 INFO:teuthology.orchestra.run.smithi035.stdout:(25/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 1.8 MB/s | 94 kB 00:00 2022-11-27T15:12:15.747 INFO:teuthology.orchestra.run.smithi085.stdout:(109/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 557 kB/s | 47 kB 00:00 2022-11-27T15:12:15.795 INFO:teuthology.orchestra.run.smithi085.stdout:(110/133): python3-oauthlib-2.1.0-1.el8.noarch. 1.9 MB/s | 155 kB 00:00 2022-11-27T15:12:15.823 INFO:teuthology.orchestra.run.smithi085.stdout:(111/133): python3-cryptography-3.2.1-5.el8.x86 4.1 MB/s | 559 kB 00:00 2022-11-27T15:12:15.857 INFO:teuthology.orchestra.run.smithi085.stdout:(112/133): python3-pycparser-2.14-14.el8.noarch 1.0 MB/s | 109 kB 00:00 2022-11-27T15:12:15.890 INFO:teuthology.orchestra.run.smithi085.stdout:(113/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.8 MB/s | 560 kB 00:00 2022-11-27T15:12:15.908 INFO:teuthology.orchestra.run.smithi035.stdout:(26/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 3.1 MB/s | 18 MB 00:05 2022-11-27T15:12:15.916 INFO:teuthology.orchestra.run.smithi085.stdout:(114/133): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.9 MB/s | 171 kB 00:00 2022-11-27T15:12:15.933 INFO:teuthology.orchestra.run.smithi035.stdout:(27/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 807 kB/s | 175 kB 00:00 2022-11-27T15:12:15.949 INFO:teuthology.orchestra.run.smithi085.stdout:(115/133): python3-jwt-1.6.1-2.el8.noarch.rpm 469 kB/s | 43 kB 00:00 2022-11-27T15:12:15.967 INFO:teuthology.orchestra.run.smithi035.stdout:(28/133): ceph-grafana-dashboards-18.0.0-594.g6 393 kB/s | 23 kB 00:00 2022-11-27T15:12:15.974 INFO:teuthology.orchestra.run.smithi085.stdout:(116/133): zlib-devel-1.2.11-18.el8_5.x86_64.rp 688 kB/s | 58 kB 00:00 2022-11-27T15:12:15.992 INFO:teuthology.orchestra.run.smithi035.stdout:(29/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 2.0 MB/s | 118 kB 00:00 2022-11-27T15:12:16.000 INFO:teuthology.orchestra.run.smithi085.stdout:(117/133): elfutils-libelf-devel-0.186-1.el8.x8 734 kB/s | 60 kB 00:00 2022-11-27T15:12:16.041 INFO:teuthology.orchestra.run.smithi085.stdout:(118/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.6 MB/s | 238 kB 00:00 2022-11-27T15:12:16.067 INFO:teuthology.orchestra.run.smithi085.stdout:(119/133): libaio-devel-0.3.112-1.el8.x86_64.rp 203 kB/s | 19 kB 00:00 2022-11-27T15:12:16.076 INFO:teuthology.orchestra.run.smithi035.stdout:(30/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 9.0 MB/s | 3.5 MB 00:00 2022-11-27T15:12:16.092 INFO:teuthology.orchestra.run.smithi085.stdout:(120/133): libuuid-devel-2.32.1-35.el8.x86_64.r 1.1 MB/s | 98 kB 00:00 2022-11-27T15:12:16.126 INFO:teuthology.orchestra.run.smithi035.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 4.6 MB/s | 234 kB 00:00 2022-11-27T15:12:16.134 INFO:teuthology.orchestra.run.smithi085.stdout:(121/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2022-11-27T15:12:16.159 INFO:teuthology.orchestra.run.smithi085.stdout:(122/133): python3-requests-oauthlib-1.0.0-1.el 464 kB/s | 43 kB 00:00 2022-11-27T15:12:16.184 INFO:teuthology.orchestra.run.smithi035.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 807 kB/s | 47 kB 00:00 2022-11-27T15:12:16.185 INFO:teuthology.orchestra.run.smithi085.stdout:(123/133): xfsprogs-devel-5.0.0-10.el8.x86_64.r 794 kB/s | 72 kB 00:00 2022-11-27T15:12:16.227 INFO:teuthology.orchestra.run.smithi085.stdout:(124/133): libcom_err-devel-1.45.6-4.el8.x86_64 419 kB/s | 39 kB 00:00 2022-11-27T15:12:16.243 INFO:teuthology.orchestra.run.smithi035.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 247 kB/s | 14 kB 00:00 2022-11-27T15:12:16.251 INFO:teuthology.orchestra.run.smithi085.stdout:(125/133): keyutils-libs-devel-1.5.10-9.el8.x86 732 kB/s | 48 kB 00:00 2022-11-27T15:12:16.293 INFO:teuthology.orchestra.run.smithi085.stdout:(126/133): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2022-11-27T15:12:16.310 INFO:teuthology.orchestra.run.smithi035.stdout:(34/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 4.8 MB/s | 1.7 MB 00:00 2022-11-27T15:12:16.328 INFO:teuthology.orchestra.run.smithi085.stdout:(127/133): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.3 MB/s | 644 kB 00:00 2022-11-27T15:12:16.336 INFO:teuthology.orchestra.run.smithi035.stdout:(35/133): ceph-volume-18.0.0-594.g697b0d99.el8. 2.8 MB/s | 263 kB 00:00 2022-11-27T15:12:16.361 INFO:teuthology.orchestra.run.smithi035.stdout:(36/133): cephadm-18.0.0-594.g697b0d99.el8.noar 1.6 MB/s | 81 kB 00:00 2022-11-27T15:12:16.369 INFO:teuthology.orchestra.run.smithi085.stdout:(128/133): openssl-devel-1.1.1k-7.el8_6.x86_64. 11 MB/s | 2.3 MB 00:00 2022-11-27T15:12:16.386 INFO:teuthology.orchestra.run.smithi035.stdout:(37/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 309 kB/s | 15 kB 00:00 2022-11-27T15:12:16.420 INFO:teuthology.orchestra.run.smithi085.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 14 MB/s | 709 kB 00:00 2022-11-27T15:12:16.478 INFO:teuthology.orchestra.run.smithi085.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 1.5 MB 00:00 2022-11-27T15:12:16.520 INFO:teuthology.orchestra.run.smithi085.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2022-11-27T15:12:16.553 INFO:teuthology.orchestra.run.smithi035.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 353 kB/s | 59 kB 00:00 2022-11-27T15:12:16.628 INFO:teuthology.orchestra.run.smithi035.stdout:(39/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2022-11-27T15:12:16.653 INFO:teuthology.orchestra.run.smithi035.stdout:(40/133): libunwind-1.3.1-3.el8.x86_64.rpm 750 kB/s | 75 kB 00:00 2022-11-27T15:12:16.703 INFO:teuthology.orchestra.run.smithi035.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 874 kB/s | 44 kB 00:00 2022-11-27T15:12:16.768 INFO:teuthology.orchestra.run.smithi035.stdout:(42/133): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2022-11-27T15:12:16.803 INFO:teuthology.orchestra.run.smithi085.stdout:(132/133): librbd1-18.0.0-594.g697b0d99.el8.x86 7.5 MB/s | 3.6 MB 00:00 2022-11-27T15:12:16.829 INFO:teuthology.orchestra.run.smithi035.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 544 kB/s | 33 kB 00:00 2022-11-27T15:12:16.854 INFO:teuthology.orchestra.run.smithi035.stdout:(44/133): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2022-11-27T15:12:16.879 INFO:teuthology.orchestra.run.smithi085.stdout:(133/133): librados2-18.0.0-594.g697b0d99.el8.x 6.2 MB/s | 3.6 MB 00:00 2022-11-27T15:12:16.879 INFO:teuthology.orchestra.run.smithi035.stdout:(45/133): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2022-11-27T15:12:16.882 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:12:16.882 INFO:teuthology.orchestra.run.smithi085.stdout:Total 18 MB/s | 219 MB 00:12 2022-11-27T15:12:16.963 INFO:teuthology.orchestra.run.smithi035.stdout:(46/133): ceph-mgr-diskprediction-local-18.0.0- 7.6 MB/s | 7.4 MB 00:00 2022-11-27T15:12:17.021 INFO:teuthology.orchestra.run.smithi035.stdout:(47/133): python3-cheroot-8.5.2-1.el8.noarch.rp 1.0 MB/s | 173 kB 00:00 2022-11-27T15:12:17.055 INFO:teuthology.orchestra.run.smithi035.stdout:(48/133): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2022-11-27T15:12:17.089 INFO:teuthology.orchestra.run.smithi035.stdout:(49/133): python3-jaraco-6.2-6.el8.noarch.rpm 164 kB/s | 11 kB 00:00 2022-11-27T15:12:17.114 INFO:teuthology.orchestra.run.smithi035.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 299 kB/s | 18 kB 00:00 2022-11-27T15:12:17.140 INFO:teuthology.orchestra.run.smithi035.stdout:(51/133): python3-google-auth-1.1.1-10.el8.noar 461 kB/s | 81 kB 00:00 2022-11-27T15:12:17.165 INFO:teuthology.orchestra.run.smithi035.stdout:(52/133): python3-logutils-0.3.5-11.el8.noarch. 970 kB/s | 49 kB 00:00 2022-11-27T15:12:17.190 INFO:teuthology.orchestra.run.smithi035.stdout:(53/133): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2022-11-27T15:12:17.216 INFO:teuthology.orchestra.run.smithi035.stdout:(54/133): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2022-11-27T15:12:17.509 INFO:teuthology.orchestra.run.smithi035.stdout:(55/133): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2022-11-27T15:12:17.599 INFO:teuthology.orchestra.run.smithi035.stdout:(56/133): python3-portend-2.6-1.el8.noarch.rpm 41 kB/s | 16 kB 00:00 2022-11-27T15:12:17.625 INFO:teuthology.orchestra.run.smithi035.stdout:(57/133): python3-repoze-lru-0.7-6.el8.noarch.r 291 kB/s | 34 kB 00:00 2022-11-27T15:12:17.650 INFO:teuthology.orchestra.run.smithi035.stdout:(58/133): python3-pecan-1.3.2-9.el8.noarch.rpm 616 kB/s | 283 kB 00:00 2022-11-27T15:12:17.675 INFO:teuthology.orchestra.run.smithi035.stdout:(59/133): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2022-11-27T15:12:17.700 INFO:teuthology.orchestra.run.smithi035.stdout:(60/133): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2022-11-27T15:12:17.717 INFO:teuthology.orchestra.run.smithi035.stdout:(61/133): python3-simplegeneric-0.8.1-17.el8.no 279 kB/s | 19 kB 00:00 2022-11-27T15:12:17.734 INFO:teuthology.orchestra.run.smithi035.stdout:(62/133): python3-singledispatch-3.4.0.3-18.el8 407 kB/s | 24 kB 00:00 2022-11-27T15:12:17.751 INFO:teuthology.orchestra.run.smithi035.stdout:(63/133): python3-tempora-1.14.1-5.el8.noarch.r 579 kB/s | 29 kB 00:00 2022-11-27T15:12:17.769 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2022-11-27T15:12:17.810 INFO:teuthology.orchestra.run.smithi035.stdout:(64/133): python3-trustme-0.6.0-4.el8.noarch.rp 299 kB/s | 27 kB 00:00 2022-11-27T15:12:17.835 INFO:teuthology.orchestra.run.smithi035.stdout:(65/133): python3-waitress-1.2.1-2.el8.1.noarch 2.3 MB/s | 241 kB 00:00 2022-11-27T15:12:17.861 INFO:teuthology.orchestra.run.smithi035.stdout:(66/133): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2022-11-27T15:12:17.877 INFO:teuthology.orchestra.run.smithi035.stdout:(67/133): python3-websocket-client-0.56.0-5.el8 912 kB/s | 61 kB 00:00 2022-11-27T15:12:17.911 INFO:teuthology.orchestra.run.smithi035.stdout:(68/133): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2022-11-27T15:12:17.928 INFO:teuthology.orchestra.run.smithi035.stdout:(69/133): python3-zc-lockfile-2.0-2.el8.noarch. 343 kB/s | 23 kB 00:00 2022-11-27T15:12:17.929 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2022-11-27T15:12:17.930 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2022-11-27T15:12:17.953 INFO:teuthology.orchestra.run.smithi035.stdout:(70/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2022-11-27T15:12:18.040 INFO:teuthology.orchestra.run.smithi035.stdout:(71/133): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2022-11-27T15:12:18.071 INFO:teuthology.orchestra.run.smithi035.stdout:(72/133): fuse-2.9.7-15.el8.x86_64.rpm 709 kB/s | 83 kB 00:00 2022-11-27T15:12:18.112 INFO:teuthology.orchestra.run.smithi035.stdout:(73/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.2 MB/s | 87 kB 00:00 2022-11-27T15:12:18.138 INFO:teuthology.orchestra.run.smithi035.stdout:(74/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.3 MB/s | 220 kB 00:00 2022-11-27T15:12:18.179 INFO:teuthology.orchestra.run.smithi035.stdout:(75/133): thrift-0.13.0-2.el8.x86_64.rpm 5.7 MB/s | 1.7 MB 00:00 2022-11-27T15:12:18.213 INFO:teuthology.orchestra.run.smithi035.stdout:(76/133): libselinux-devel-2.9-5.el8.x86_64.rpm 1.9 MB/s | 200 kB 00:00 2022-11-27T15:12:18.238 INFO:teuthology.orchestra.run.smithi035.stdout:(77/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 180 kB/s | 18 kB 00:00 2022-11-27T15:12:18.297 INFO:teuthology.orchestra.run.smithi035.stdout:(78/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 408 kB/s | 47 kB 00:00 2022-11-27T15:12:18.322 INFO:teuthology.orchestra.run.smithi035.stdout:(79/133): python3-oauthlib-2.1.0-1.el8.noarch.r 1.8 MB/s | 155 kB 00:00 2022-11-27T15:12:18.364 INFO:teuthology.orchestra.run.smithi035.stdout:(80/133): python3-cryptography-3.2.1-5.el8.x86_ 3.6 MB/s | 559 kB 00:00 2022-11-27T15:12:18.398 INFO:teuthology.orchestra.run.smithi035.stdout:(81/133): python3-pycparser-2.14-14.el8.noarch. 1.1 MB/s | 109 kB 00:00 2022-11-27T15:12:18.431 INFO:teuthology.orchestra.run.smithi035.stdout:(82/133): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.6 MB/s | 171 kB 00:00 2022-11-27T15:12:18.465 INFO:teuthology.orchestra.run.smithi035.stdout:(83/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.9 MB/s | 560 kB 00:00 2022-11-27T15:12:18.531 INFO:teuthology.orchestra.run.smithi035.stdout:(84/133): python3-jwt-1.6.1-2.el8.noarch.rpm 322 kB/s | 43 kB 00:00 2022-11-27T15:12:18.565 INFO:teuthology.orchestra.run.smithi035.stdout:(85/133): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 435 kB/s | 58 kB 00:00 2022-11-27T15:12:18.591 INFO:teuthology.orchestra.run.smithi035.stdout:(86/133): elfutils-libelf-devel-0.186-1.el8.x86 487 kB/s | 60 kB 00:00 2022-11-27T15:12:18.641 INFO:teuthology.orchestra.run.smithi035.stdout:(87/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2022-11-27T15:12:18.666 INFO:teuthology.orchestra.run.smithi035.stdout:(88/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 249 kB/s | 19 kB 00:00 2022-11-27T15:12:18.691 INFO:teuthology.orchestra.run.smithi035.stdout:(89/133): libuuid-devel-2.32.1-35.el8.x86_64.rp 980 kB/s | 98 kB 00:00 2022-11-27T15:12:18.733 INFO:teuthology.orchestra.run.smithi035.stdout:(90/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2022-11-27T15:12:18.792 INFO:teuthology.orchestra.run.smithi035.stdout:(91/133): python3-requests-oauthlib-1.0.0-1.el8 340 kB/s | 43 kB 00:00 2022-11-27T15:12:18.817 INFO:teuthology.orchestra.run.smithi035.stdout:(92/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 582 kB/s | 72 kB 00:00 2022-11-27T15:12:18.859 INFO:teuthology.orchestra.run.smithi035.stdout:(93/133): libcom_err-devel-1.45.6-4.el8.x86_64. 307 kB/s | 39 kB 00:00 2022-11-27T15:12:18.884 INFO:teuthology.orchestra.run.smithi035.stdout:(94/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 732 kB/s | 48 kB 00:00 2022-11-27T15:12:18.927 INFO:teuthology.orchestra.run.smithi035.stdout:(95/133): NetworkManager-initscripts-updown-1.3 3.3 MB/s | 138 kB 00:00 2022-11-27T15:12:18.952 INFO:teuthology.orchestra.run.smithi035.stdout:(96/133): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.8 MB/s | 644 kB 00:00 2022-11-27T15:12:19.027 INFO:teuthology.orchestra.run.smithi035.stdout:(97/133): openssl-devel-1.1.1k-7.el8_6.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2022-11-27T15:12:19.070 INFO:teuthology.orchestra.run.smithi035.stdout:(98/133): bison-3.0.4-10.el8.x86_64.rpm 5.8 MB/s | 688 kB 00:00 2022-11-27T15:12:19.089 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2022-11-27T15:12:19.093 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2022-11-27T15:12:19.103 INFO:teuthology.orchestra.run.smithi035.stdout:(99/133): postgresql-contrib-10.21-2.module+el8 4.5 MB/s | 808 kB 00:00 2022-11-27T15:12:19.153 INFO:teuthology.orchestra.run.smithi035.stdout:(100/133): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2022-11-27T15:12:19.211 INFO:teuthology.orchestra.run.smithi035.stdout:(101/133): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2022-11-27T15:12:19.270 INFO:teuthology.orchestra.run.smithi035.stdout:(102/133): python3-markupsafe-0.23-19.el8.x86_6 667 kB/s | 39 kB 00:00 2022-11-27T15:12:19.337 INFO:teuthology.orchestra.run.smithi035.stdout:(103/133): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2022-11-27T15:12:19.362 INFO:teuthology.orchestra.run.smithi035.stdout:(104/133): python3-prettytable-0.7.2-14.el8.noa 479 kB/s | 44 kB 00:00 2022-11-27T15:12:19.437 INFO:teuthology.orchestra.run.smithi035.stdout:(105/133): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2022-11-27T15:12:19.499 INFO:teuthology.orchestra.run.smithi035.stdout:(106/133): librdkafka-0.11.4-3.el8.x86_64.rpm 2.1 MB/s | 354 kB 00:00 2022-11-27T15:12:19.538 INFO:teuthology.orchestra.run.smithi035.stdout:(107/133): python3-jinja2-2.10.1-3.el8.noarch.r 3.0 MB/s | 538 kB 00:00 2022-11-27T15:12:19.572 INFO:teuthology.orchestra.run.smithi035.stdout:(108/133): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2022-11-27T15:12:19.597 INFO:teuthology.orchestra.run.smithi035.stdout:(109/133): python3-numpy-f2py-1.14.3-10.el8.x86 2.3 MB/s | 225 kB 00:00 2022-11-27T15:12:19.639 INFO:teuthology.orchestra.run.smithi035.stdout:(110/133): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2022-11-27T15:12:19.681 INFO:teuthology.orchestra.run.smithi035.stdout:(111/133): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2022-11-27T15:12:19.765 INFO:teuthology.orchestra.run.smithi035.stdout:(112/133): socat-1.7.4.1-1.el8.x86_64.rpm 3.8 MB/s | 323 kB 00:00 2022-11-27T15:12:19.840 INFO:teuthology.orchestra.run.smithi035.stdout:(113/133): openblas-threads-0.3.15-3.el8.x86_64 16 MB/s | 4.9 MB 00:00 2022-11-27T15:12:19.873 INFO:teuthology.orchestra.run.smithi035.stdout:(114/133): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2022-11-27T15:12:19.940 INFO:teuthology.orchestra.run.smithi035.stdout:(115/133): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2022-11-27T15:12:19.999 INFO:teuthology.orchestra.run.smithi035.stdout:(116/133): jq-1.6-3.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2022-11-27T15:12:20.058 INFO:teuthology.orchestra.run.smithi035.stdout:(117/133): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2022-11-27T15:12:20.133 INFO:teuthology.orchestra.run.smithi035.stdout:(118/133): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2022-11-27T15:12:20.158 INFO:teuthology.orchestra.run.smithi035.stdout:(119/133): python3-pytz-2017.2-9.el8.noarch.rpm 542 kB/s | 54 kB 00:00 2022-11-27T15:12:20.229 INFO:teuthology.orchestra.run.smithi035.stdout:(120/133): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.3 MB/s | 126 kB 00:00 2022-11-27T15:12:20.259 INFO:teuthology.orchestra.run.smithi035.stdout:(121/133): python3-jsonpointer-1.10-11.el8.noar 199 kB/s | 20 kB 00:00 2022-11-27T15:12:20.301 INFO:teuthology.orchestra.run.smithi035.stdout:(122/133): libpq-13.5-1.el8.x86_64.rpm 2.7 MB/s | 198 kB 00:00 2022-11-27T15:12:20.317 INFO:teuthology.orchestra.run.smithi035.stdout:(123/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2022-11-27T15:12:20.351 INFO:teuthology.orchestra.run.smithi035.stdout:(124/133): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2022-11-27T15:12:20.376 INFO:teuthology.orchestra.run.smithi035.stdout:(125/133): python3-jsonpatch-1.21-2.el8.noarch. 453 kB/s | 27 kB 00:00 2022-11-27T15:12:20.418 INFO:teuthology.orchestra.run.smithi035.stdout:(126/133): uuid-1.6.2-43.el8.x86_64.rpm 957 kB/s | 64 kB 00:00 2022-11-27T15:12:20.466 INFO:teuthology.orchestra.run.smithi035.stdout:(127/133): python3-pyasn1-modules-0.3.7-6.el8.n 1.2 MB/s | 110 kB 00:00 2022-11-27T15:12:20.636 INFO:teuthology.orchestra.run.smithi035.stdout:(128/133): python3-scipy-1.0.0-21.module+el8.5. 20 MB/s | 14 MB 00:00 2022-11-27T15:12:20.686 INFO:teuthology.orchestra.run.smithi035.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 14 MB/s | 709 kB 00:00 2022-11-27T15:12:20.753 INFO:teuthology.orchestra.run.smithi035.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 22 MB/s | 1.5 MB 00:00 2022-11-27T15:12:20.811 INFO:teuthology.orchestra.run.smithi035.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2022-11-27T15:12:21.062 INFO:teuthology.orchestra.run.smithi035.stdout:(132/133): librbd1-18.0.0-594.g697b0d99.el8.x86 6.0 MB/s | 3.6 MB 00:00 2022-11-27T15:12:21.128 INFO:teuthology.orchestra.run.smithi035.stdout:(133/133): librados2-18.0.0-594.g697b0d99.el8.x 5.1 MB/s | 3.6 MB 00:00 2022-11-27T15:12:21.132 INFO:teuthology.orchestra.run.smithi035.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:12:21.132 INFO:teuthology.orchestra.run.smithi035.stdout:Total 15 MB/s | 219 MB 00:14 2022-11-27T15:12:22.053 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction check 2022-11-27T15:12:22.230 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction check succeeded. 2022-11-27T15:12:22.231 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction test 2022-11-27T15:12:23.490 INFO:teuthology.orchestra.run.smithi035.stdout:Transaction test succeeded. 2022-11-27T15:12:23.495 INFO:teuthology.orchestra.run.smithi035.stdout:Running transaction 2022-11-27T15:12:23.989 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2022-11-27T15:12:24.212 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T15:12:24.426 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T15:12:24.598 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-11-27T15:12:24.798 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-11-27T15:12:24.942 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T15:12:25.005 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T15:12:25.184 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 5/138 2022-11-27T15:12:25.342 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T15:12:25.718 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T15:12:25.958 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T15:12:26.334 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T15:12:26.551 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T15:12:26.671 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T15:12:26.875 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T15:12:26.968 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T15:12:27.139 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T15:12:27.344 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-11-27T15:12:27.545 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-11-27T15:12:27.886 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-11-27T15:12:28.079 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-11-27T15:12:28.234 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T15:12:28.333 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T15:12:28.491 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T15:12:28.564 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T15:12:28.714 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 17/138 2022-11-27T15:12:28.880 INFO:teuthology.orchestra.run.smithi035.stdout: Preparing : 1/1 2022-11-27T15:12:28.883 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T15:12:29.016 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T15:12:29.116 INFO:teuthology.orchestra.run.smithi035.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T15:12:29.142 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T15:12:29.336 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T15:12:29.549 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-11-27T15:12:29.673 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-11-27T15:12:29.756 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T15:12:29.893 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/138 2022-11-27T15:12:29.941 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T15:12:30.068 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/138 2022-11-27T15:12:30.171 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/138 2022-11-27T15:12:30.368 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T15:12:30.539 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T15:12:30.726 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T15:12:30.783 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T15:12:31.034 INFO:teuthology.orchestra.run.smithi035.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T15:12:31.403 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T15:12:31.448 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T15:12:31.602 INFO:teuthology.orchestra.run.smithi035.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T15:12:31.663 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 22/138 2022-11-27T15:12:31.716 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T15:12:31.810 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 23/138 2022-11-27T15:12:31.876 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T15:12:31.995 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T15:12:32.115 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/138 2022-11-27T15:12:32.150 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T15:12:32.367 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-11-27T15:12:32.397 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/138 2022-11-27T15:12:32.555 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-11-27T15:12:32.659 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-11-27T15:12:32.839 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/138 2022-11-27T15:12:32.953 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-11-27T15:12:32.976 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/138 2022-11-27T15:12:33.197 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-11-27T15:12:33.200 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/138 2022-11-27T15:12:33.335 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T15:12:33.475 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T15:12:33.961 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T15:12:34.049 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T15:12:34.108 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 30/138 2022-11-27T15:12:34.249 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 17/138 2022-11-27T15:12:34.462 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/138 2022-11-27T15:12:34.519 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/138 2022-11-27T15:12:34.668 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/138 2022-11-27T15:12:34.816 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 20/138 2022-11-27T15:12:34.818 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/138 2022-11-27T15:12:34.951 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-11-27T15:12:35.016 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T15:12:35.093 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-11-27T15:12:35.234 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-11-27T15:12:35.895 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-11-27T15:12:36.084 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-11-27T15:12:36.733 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T15:12:36.742 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-11-27T15:12:36.926 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : openblas-0.3.15-3.el8.x86_64 24/138 2022-11-27T15:12:37.058 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/138 2022-11-27T15:12:37.269 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-11-27T15:12:37.483 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-11-27T15:12:37.647 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 24/138 2022-11-27T15:12:37.686 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 37/138 2022-11-27T15:12:37.885 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 38/138 2022-11-27T15:12:37.985 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/138 2022-11-27T15:12:38.068 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-11-27T15:12:38.228 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 40/138 2022-11-27T15:12:38.259 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-11-27T15:12:38.376 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 41/138 2022-11-27T15:12:38.482 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/138 2022-11-27T15:12:38.555 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/138 2022-11-27T15:12:38.662 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/138 2022-11-27T15:12:38.761 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 43/138 2022-11-27T15:12:38.935 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/138 2022-11-27T15:12:38.992 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 44/138 2022-11-27T15:12:39.151 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-11-27T15:12:39.405 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 45/138 2022-11-27T15:12:39.726 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 46/138 2022-11-27T15:12:39.925 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 47/138 2022-11-27T15:12:40.185 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 48/138 2022-11-27T15:12:40.436 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 49/138 2022-11-27T15:12:40.738 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 50/138 2022-11-27T15:12:40.964 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 51/138 2022-11-27T15:12:40.996 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-11-27T15:12:41.158 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 52/138 2022-11-27T15:12:41.373 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/138 2022-11-27T15:12:41.408 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 53/138 2022-11-27T15:12:41.559 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 54/138 2022-11-27T15:12:41.875 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/138 2022-11-27T15:12:42.069 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/138 2022-11-27T15:12:42.337 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 57/138 2022-11-27T15:12:42.400 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/138 2022-11-27T15:12:42.585 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 58/138 2022-11-27T15:12:42.754 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/138 2022-11-27T15:12:42.857 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 59/138 2022-11-27T15:12:43.019 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/138 2022-11-27T15:12:43.057 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : fuse-2.9.7-15.el8.x86_64 60/138 2022-11-27T15:12:43.256 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T15:12:43.289 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-11-27T15:12:43.344 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T15:12:43.453 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-11-27T15:12:43.555 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T15:12:43.656 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T15:12:43.667 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 37/138 2022-11-27T15:12:43.838 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : jq-1.6-3.el8.x86_64 63/138 2022-11-27T15:12:43.897 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 38/138 2022-11-27T15:12:44.014 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 64/138 2022-11-27T15:12:44.118 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-11-27T15:12:44.280 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 65/138 2022-11-27T15:12:44.342 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/138 2022-11-27T15:12:44.411 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/138 2022-11-27T15:12:44.498 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : automake-1.16.1-7.el8.noarch 66/138 2022-11-27T15:12:44.599 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 41/138 2022-11-27T15:12:44.668 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 41/138 2022-11-27T15:12:44.708 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 67/138 2022-11-27T15:12:44.842 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : jq-1.6-3.el8.x86_64 42/138 2022-11-27T15:12:44.889 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-11-27T15:12:45.025 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 43/138 2022-11-27T15:12:45.052 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T15:12:45.281 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 44/138 2022-11-27T15:12:45.326 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-11-27T15:12:45.526 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : automake-1.16.1-7.el8.noarch 45/138 2022-11-27T15:12:45.547 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-11-27T15:12:45.706 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 46/138 2022-11-27T15:12:45.727 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-11-27T15:12:45.821 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 47/138 2022-11-27T15:12:45.875 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-11-27T15:12:46.013 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/138 2022-11-27T15:12:46.065 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-11-27T15:12:46.242 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 49/138 2022-11-27T15:12:46.284 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-11-27T15:12:46.493 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 50/138 2022-11-27T15:12:46.517 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-11-27T15:12:46.692 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-11-27T15:12:46.792 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/138 2022-11-27T15:12:46.836 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-11-27T15:12:47.042 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-11-27T15:12:47.251 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/138 2022-11-27T15:12:47.267 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-11-27T15:12:47.445 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-11-27T15:12:47.571 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/138 2022-11-27T15:12:47.700 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-11-27T15:12:47.804 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/138 2022-11-27T15:12:47.912 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-11-27T15:12:48.095 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-11-27T15:12:48.096 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/138 2022-11-27T15:12:48.310 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-11-27T15:12:48.356 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/138 2022-11-27T15:12:48.694 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/138 2022-11-27T15:12:49.004 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/138 2022-11-27T15:12:49.227 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/138 2022-11-27T15:12:49.353 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-11-27T15:12:49.543 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/138 2022-11-27T15:12:49.625 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-11-27T15:12:49.938 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-11-27T15:12:49.954 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 61/138 2022-11-27T15:12:50.229 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-11-27T15:12:50.327 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/138 2022-11-27T15:12:50.531 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-11-27T15:12:50.555 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/138 2022-11-27T15:12:50.696 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-11-27T15:12:50.864 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 64/138 2022-11-27T15:12:51.116 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/138 2022-11-27T15:12:51.392 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/138 2022-11-27T15:12:51.484 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-11-27T15:12:51.618 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : fuse-2.9.7-15.el8.x86_64 67/138 2022-11-27T15:12:51.646 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T15:12:51.753 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T15:12:51.800 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-11-27T15:12:51.939 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 94/138 2022-11-27T15:12:52.021 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T15:12:52.329 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-11-27T15:12:52.548 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-11-27T15:12:52.754 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-11-27T15:12:52.947 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-11-27T15:12:53.169 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-11-27T15:12:53.413 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-11-27T15:12:53.628 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-11-27T15:12:53.835 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-11-27T15:12:53.980 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-11-27T15:12:54.222 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-11-27T15:12:54.448 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-11-27T15:12:54.646 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-11-27T15:12:54.924 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-11-27T15:12:55.215 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-11-27T15:12:55.424 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-11-27T15:12:55.667 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-11-27T15:12:55.680 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T15:12:55.888 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T15:12:56.443 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T15:12:56.715 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T15:12:56.795 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T15:12:56.810 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-11-27T15:12:56.938 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T15:12:57.120 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-11-27T15:12:57.394 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-11-27T15:12:57.733 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-11-27T15:12:58.008 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-11-27T15:12:58.218 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-11-27T15:12:59.051 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-11-27T15:12:59.214 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T15:12:59.316 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T15:12:59.557 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 94/138 2022-11-27T15:13:03.678 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T15:13:03.923 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T15:13:04.516 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T15:13:04.842 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T15:13:04.934 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T15:13:05.055 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T15:13:15.605 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T15:13:15.605 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /sys 2022-11-27T15:13:15.606 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /proc 2022-11-27T15:13:15.606 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /dev 2022-11-27T15:13:15.606 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /run 2022-11-27T15:13:15.606 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /mnt 2022-11-27T15:13:15.606 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /var/tmp 2022-11-27T15:13:15.607 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /home 2022-11-27T15:13:15.607 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /tmp 2022-11-27T15:13:15.607 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /dev 2022-11-27T15:13:15.607 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:15.977 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T15:13:17.457 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T15:13:17.647 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T15:13:17.863 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T15:13:18.026 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T15:13:18.407 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T15:13:18.593 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T15:13:18.610 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T15:13:20.120 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T15:13:20.289 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T15:13:20.406 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 103/138 2022-11-27T15:13:20.572 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 104/138 2022-11-27T15:13:20.725 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T15:13:20.932 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T15:13:21.231 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T15:13:22.187 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T15:13:22.325 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T15:13:22.413 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T15:13:22.812 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 108/138 2022-11-27T15:13:22.996 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T15:13:23.031 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T15:13:23.178 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T15:13:23.187 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T15:13:24.750 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T15:13:24.750 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /sys 2022-11-27T15:13:24.751 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /proc 2022-11-27T15:13:24.751 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /dev 2022-11-27T15:13:24.752 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /run 2022-11-27T15:13:24.752 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /mnt 2022-11-27T15:13:24.753 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /var/tmp 2022-11-27T15:13:24.753 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /home 2022-11-27T15:13:24.753 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /tmp 2022-11-27T15:13:24.754 INFO:teuthology.orchestra.run.smithi035.stdout:skipping the directory /dev 2022-11-27T15:13:24.754 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:24.890 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 111/138 2022-11-27T15:13:25.080 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T15:13:25.114 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T15:13:25.153 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T15:13:25.266 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T15:13:25.308 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T15:13:25.475 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T15:13:25.820 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T15:13:25.975 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T15:13:26.738 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T15:13:26.934 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T15:13:27.165 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T15:13:27.361 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T15:13:27.767 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T15:13:27.936 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T15:13:27.955 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T15:13:29.790 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T15:13:29.953 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T15:13:30.059 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 103/138 2022-11-27T15:13:30.267 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 104/138 2022-11-27T15:13:30.452 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T15:13:30.665 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T15:13:30.966 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T15:13:31.061 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T15:13:31.342 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 116/138 2022-11-27T15:13:31.490 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T15:13:31.680 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T15:13:31.991 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T15:13:32.005 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 118/138 2022-11-27T15:13:32.185 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T15:13:32.280 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T15:13:32.685 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 108/138 2022-11-27T15:13:32.829 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 119/138 2022-11-27T15:13:32.893 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T15:13:32.930 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T15:13:33.084 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 120/138 2022-11-27T15:13:33.127 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T15:13:33.139 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T15:13:33.261 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 121/138 2022-11-27T15:13:33.469 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 122/138 2022-11-27T15:13:33.607 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-11-27T15:13:34.967 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 111/138 2022-11-27T15:13:35.178 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T15:13:35.213 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T15:13:35.385 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T15:13:35.429 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T15:13:35.585 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T15:13:35.632 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T15:13:35.876 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T15:13:35.890 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T15:13:35.954 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T15:13:36.098 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 125/138 2022-11-27T15:13:36.127 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T15:13:36.339 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 126/138 2022-11-27T15:13:36.614 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 127/138 2022-11-27T15:13:36.971 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-11-27T15:13:37.267 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-11-27T15:13:37.415 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T15:13:37.424 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T15:13:37.574 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/138 2022-11-27T15:13:37.806 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T15:13:37.881 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T15:13:38.072 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T15:13:38.087 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T15:13:38.267 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T15:13:38.267 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T15:13:38.466 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T15:13:38.467 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T15:13:38.708 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T15:13:38.708 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T15:13:38.973 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T15:13:39.156 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T15:13:39.708 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T15:13:39.708 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/138 2022-11-27T15:13:39.709 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/138 2022-11-27T15:13:39.709 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/138 2022-11-27T15:13:39.709 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/138 2022-11-27T15:13:39.709 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/138 2022-11-27T15:13:39.710 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/138 2022-11-27T15:13:39.710 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T15:13:39.710 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T15:13:39.710 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T15:13:39.711 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T15:13:39.711 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/138 2022-11-27T15:13:39.711 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/138 2022-11-27T15:13:39.711 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/138 2022-11-27T15:13:39.712 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/138 2022-11-27T15:13:39.712 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T15:13:39.712 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/138 2022-11-27T15:13:39.712 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/138 2022-11-27T15:13:39.712 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/138 2022-11-27T15:13:39.713 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/138 2022-11-27T15:13:39.713 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/138 2022-11-27T15:13:39.713 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/138 2022-11-27T15:13:39.713 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/138 2022-11-27T15:13:39.714 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/138 2022-11-27T15:13:39.714 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/138 2022-11-27T15:13:39.714 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/138 2022-11-27T15:13:39.714 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/138 2022-11-27T15:13:39.715 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/138 2022-11-27T15:13:39.729 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/138 2022-11-27T15:13:39.729 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/138 2022-11-27T15:13:39.729 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/138 2022-11-27T15:13:39.729 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/138 2022-11-27T15:13:39.730 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/138 2022-11-27T15:13:39.730 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/138 2022-11-27T15:13:39.731 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/138 2022-11-27T15:13:39.731 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/138 2022-11-27T15:13:39.732 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/138 2022-11-27T15:13:39.732 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/138 2022-11-27T15:13:39.732 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/138 2022-11-27T15:13:39.733 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-11-27T15:13:39.733 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-11-27T15:13:39.733 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-11-27T15:13:39.733 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-11-27T15:13:39.733 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-11-27T15:13:39.734 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-11-27T15:13:39.734 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-11-27T15:13:39.734 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-11-27T15:13:39.735 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-11-27T15:13:39.735 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-11-27T15:13:39.736 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-11-27T15:13:39.736 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-11-27T15:13:39.737 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-11-27T15:13:39.737 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-11-27T15:13:39.737 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-11-27T15:13:39.738 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-11-27T15:13:39.738 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-11-27T15:13:39.738 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-11-27T15:13:39.739 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-11-27T15:13:39.739 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-11-27T15:13:39.740 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-11-27T15:13:39.740 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-11-27T15:13:39.741 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-11-27T15:13:39.741 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-11-27T15:13:39.742 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-11-27T15:13:39.742 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-11-27T15:13:39.742 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-11-27T15:13:39.743 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-11-27T15:13:39.743 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-11-27T15:13:39.744 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-11-27T15:13:39.744 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T15:13:39.745 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-11-27T15:13:39.745 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-11-27T15:13:39.745 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 72/138 2022-11-27T15:13:39.746 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : bison-3.0.4-10.el8.x86_64 73/138 2022-11-27T15:13:39.746 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 74/138 2022-11-27T15:13:39.746 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 75/138 2022-11-27T15:13:39.747 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 76/138 2022-11-27T15:13:39.747 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/138 2022-11-27T15:13:39.748 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 78/138 2022-11-27T15:13:39.748 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 79/138 2022-11-27T15:13:39.748 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 80/138 2022-11-27T15:13:39.748 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 81/138 2022-11-27T15:13:39.749 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 82/138 2022-11-27T15:13:39.749 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 83/138 2022-11-27T15:13:39.749 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 84/138 2022-11-27T15:13:39.749 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 85/138 2022-11-27T15:13:39.750 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/138 2022-11-27T15:13:39.750 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 87/138 2022-11-27T15:13:39.750 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/138 2022-11-27T15:13:39.750 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : flex-2.6.1-9.el8.x86_64 89/138 2022-11-27T15:13:39.750 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 90/138 2022-11-27T15:13:39.751 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : jq-1.6-3.el8.x86_64 91/138 2022-11-27T15:13:39.751 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 92/138 2022-11-27T15:13:39.751 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : automake-1.16.1-7.el8.noarch 93/138 2022-11-27T15:13:39.751 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 94/138 2022-11-27T15:13:39.752 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 95/138 2022-11-27T15:13:39.752 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 96/138 2022-11-27T15:13:39.752 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libpq-13.5-1.el8.x86_64 97/138 2022-11-27T15:13:39.752 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 98/138 2022-11-27T15:13:39.752 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 99/138 2022-11-27T15:13:39.753 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 100/138 2022-11-27T15:13:39.753 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 101/138 2022-11-27T15:13:39.753 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/138 2022-11-27T15:13:39.754 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 103/138 2022-11-27T15:13:39.754 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 104/138 2022-11-27T15:13:39.754 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 105/138 2022-11-27T15:13:39.754 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/138 2022-11-27T15:13:39.755 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 107/138 2022-11-27T15:13:39.755 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 108/138 2022-11-27T15:13:39.755 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 109/138 2022-11-27T15:13:39.755 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 110/138 2022-11-27T15:13:39.756 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 111/138 2022-11-27T15:13:39.756 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 112/138 2022-11-27T15:13:39.756 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 113/138 2022-11-27T15:13:39.756 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 114/138 2022-11-27T15:13:39.756 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 115/138 2022-11-27T15:13:39.757 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/138 2022-11-27T15:13:39.757 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 117/138 2022-11-27T15:13:39.757 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/138 2022-11-27T15:13:39.757 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 119/138 2022-11-27T15:13:39.758 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 120/138 2022-11-27T15:13:39.758 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 121/138 2022-11-27T15:13:39.758 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 122/138 2022-11-27T15:13:39.758 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 123/138 2022-11-27T15:13:39.758 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 124/138 2022-11-27T15:13:39.759 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 125/138 2022-11-27T15:13:39.759 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 126/138 2022-11-27T15:13:39.759 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 127/138 2022-11-27T15:13:39.759 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/138 2022-11-27T15:13:39.760 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 129/138 2022-11-27T15:13:39.760 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-11-27T15:13:39.760 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 131/138 2022-11-27T15:13:39.760 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-11-27T15:13:39.760 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-11-27T15:13:39.761 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T15:13:39.761 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-11-27T15:13:39.761 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-11-27T15:13:39.761 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-11-27T15:13:41.586 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T15:13:41.685 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T15:13:41.685 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2022-11-27T15:13:41.924 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:41.924 INFO:teuthology.orchestra.run.smithi085.stdout:Upgraded: 2022-11-27T15:13:41.925 INFO:teuthology.orchestra.run.smithi085.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-11-27T15:13:41.925 INFO:teuthology.orchestra.run.smithi085.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.925 INFO:teuthology.orchestra.run.smithi085.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.926 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-11-27T15:13:41.926 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-11-27T15:13:41.926 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2022-11-27T15:13:41.926 INFO:teuthology.orchestra.run.smithi085.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-11-27T15:13:41.927 INFO:teuthology.orchestra.run.smithi085.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T15:13:41.927 INFO:teuthology.orchestra.run.smithi085.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T15:13:41.927 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.928 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.928 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.928 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.929 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:41.929 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.929 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.929 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.930 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:41.930 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:41.930 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:41.930 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:41.931 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:41.931 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.931 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.932 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:41.932 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.933 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.933 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.933 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:41.933 INFO:teuthology.orchestra.run.smithi085.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:41.934 INFO:teuthology.orchestra.run.smithi085.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.934 INFO:teuthology.orchestra.run.smithi085.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:41.934 INFO:teuthology.orchestra.run.smithi085.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-11-27T15:13:41.934 INFO:teuthology.orchestra.run.smithi085.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T15:13:41.935 INFO:teuthology.orchestra.run.smithi085.stdout: fuse-2.9.7-15.el8.x86_64 2022-11-27T15:13:41.935 INFO:teuthology.orchestra.run.smithi085.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T15:13:41.935 INFO:teuthology.orchestra.run.smithi085.stdout: jq-1.6-3.el8.x86_64 2022-11-27T15:13:41.935 INFO:teuthology.orchestra.run.smithi085.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T15:13:41.936 INFO:teuthology.orchestra.run.smithi085.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-11-27T15:13:41.936 INFO:teuthology.orchestra.run.smithi085.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T15:13:41.936 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.936 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.937 INFO:teuthology.orchestra.run.smithi085.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.937 INFO:teuthology.orchestra.run.smithi085.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-11-27T15:13:41.937 INFO:teuthology.orchestra.run.smithi085.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-11-27T15:13:41.937 INFO:teuthology.orchestra.run.smithi085.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T15:13:41.938 INFO:teuthology.orchestra.run.smithi085.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T15:13:41.938 INFO:teuthology.orchestra.run.smithi085.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-11-27T15:13:41.938 INFO:teuthology.orchestra.run.smithi085.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T15:13:41.938 INFO:teuthology.orchestra.run.smithi085.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.939 INFO:teuthology.orchestra.run.smithi085.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.939 INFO:teuthology.orchestra.run.smithi085.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T15:13:41.939 INFO:teuthology.orchestra.run.smithi085.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.939 INFO:teuthology.orchestra.run.smithi085.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-11-27T15:13:41.940 INFO:teuthology.orchestra.run.smithi085.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T15:13:41.940 INFO:teuthology.orchestra.run.smithi085.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T15:13:41.940 INFO:teuthology.orchestra.run.smithi085.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T15:13:41.940 INFO:teuthology.orchestra.run.smithi085.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-11-27T15:13:41.941 INFO:teuthology.orchestra.run.smithi085.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-11-27T15:13:41.941 INFO:teuthology.orchestra.run.smithi085.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T15:13:41.941 INFO:teuthology.orchestra.run.smithi085.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T15:13:41.941 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-0.3.15-3.el8.x86_64 2022-11-27T15:13:41.942 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-11-27T15:13:41.942 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-11-27T15:13:41.942 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-11-27T15:13:41.942 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-11-27T15:13:41.943 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-11-27T15:13:41.943 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T15:13:41.943 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T15:13:41.944 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T15:13:41.944 INFO:teuthology.orchestra.run.smithi085.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T15:13:41.944 INFO:teuthology.orchestra.run.smithi085.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T15:13:41.944 INFO:teuthology.orchestra.run.smithi085.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T15:13:41.945 INFO:teuthology.orchestra.run.smithi085.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T15:13:41.945 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T15:13:41.945 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.945 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.945 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.946 INFO:teuthology.orchestra.run.smithi085.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T15:13:41.946 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T15:13:41.946 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T15:13:41.946 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T15:13:41.947 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T15:13:41.947 INFO:teuthology.orchestra.run.smithi085.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T15:13:41.947 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T15:13:41.947 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T15:13:41.948 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T15:13:41.948 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T15:13:41.948 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T15:13:41.948 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T15:13:41.949 INFO:teuthology.orchestra.run.smithi085.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T15:13:41.949 INFO:teuthology.orchestra.run.smithi085.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T15:13:41.949 INFO:teuthology.orchestra.run.smithi085.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T15:13:41.949 INFO:teuthology.orchestra.run.smithi085.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T15:13:41.950 INFO:teuthology.orchestra.run.smithi085.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T15:13:41.950 INFO:teuthology.orchestra.run.smithi085.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T15:13:41.950 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T15:13:41.950 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T15:13:41.951 INFO:teuthology.orchestra.run.smithi085.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T15:13:41.951 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T15:13:41.951 INFO:teuthology.orchestra.run.smithi085.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T15:13:41.951 INFO:teuthology.orchestra.run.smithi085.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T15:13:41.952 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T15:13:41.952 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T15:13:41.952 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T15:13:41.952 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T15:13:41.953 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T15:13:41.953 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.953 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.953 INFO:teuthology.orchestra.run.smithi085.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T15:13:41.954 INFO:teuthology.orchestra.run.smithi085.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T15:13:41.954 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.954 INFO:teuthology.orchestra.run.smithi085.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T15:13:41.954 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T15:13:41.954 INFO:teuthology.orchestra.run.smithi085.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-11-27T15:13:41.955 INFO:teuthology.orchestra.run.smithi085.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T15:13:41.955 INFO:teuthology.orchestra.run.smithi085.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T15:13:41.955 INFO:teuthology.orchestra.run.smithi085.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T15:13:41.955 INFO:teuthology.orchestra.run.smithi085.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T15:13:41.956 INFO:teuthology.orchestra.run.smithi085.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T15:13:41.956 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T15:13:41.956 INFO:teuthology.orchestra.run.smithi085.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T15:13:41.956 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T15:13:41.957 INFO:teuthology.orchestra.run.smithi085.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T15:13:41.957 INFO:teuthology.orchestra.run.smithi085.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T15:13:41.957 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.957 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.958 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:41.958 INFO:teuthology.orchestra.run.smithi085.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T15:13:41.958 INFO:teuthology.orchestra.run.smithi085.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T15:13:41.958 INFO:teuthology.orchestra.run.smithi085.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T15:13:41.959 INFO:teuthology.orchestra.run.smithi085.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T15:13:41.959 INFO:teuthology.orchestra.run.smithi085.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T15:13:41.960 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 116/138 2022-11-27T15:13:41.960 INFO:teuthology.orchestra.run.smithi085.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-11-27T15:13:41.960 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:41.961 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2022-11-27T15:13:42.113 DEBUG:teuthology.parallel:result is None 2022-11-27T15:13:42.824 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 117/138 2022-11-27T15:13:43.107 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/138 2022-11-27T15:13:43.365 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/138 2022-11-27T15:13:43.567 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/138 2022-11-27T15:13:43.775 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/138 2022-11-27T15:13:44.003 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/138 2022-11-27T15:13:44.197 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 122/138 2022-11-27T15:13:44.376 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-11-27T15:13:46.648 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T15:13:46.942 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T15:13:46.959 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T15:13:47.220 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 125/138 2022-11-27T15:13:47.385 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 126/138 2022-11-27T15:13:47.674 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 127/138 2022-11-27T15:13:48.074 INFO:teuthology.orchestra.run.smithi035.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-11-27T15:13:48.396 INFO:teuthology.orchestra.run.smithi035.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-11-27T15:13:48.636 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/138 2022-11-27T15:13:48.716 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/138 2022-11-27T15:13:48.941 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/138 2022-11-27T15:13:48.955 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/138 2022-11-27T15:13:49.134 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-11-27T15:13:49.143 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-11-27T15:13:49.266 INFO:teuthology.orchestra.run.smithi035.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 133/138 2022-11-27T15:13:49.477 INFO:teuthology.orchestra.run.smithi035.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T15:13:49.478 INFO:teuthology.orchestra.run.smithi035.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T15:13:49.712 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T15:13:49.712 INFO:teuthology.orchestra.run.smithi035.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T15:13:49.970 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T15:13:49.970 INFO:teuthology.orchestra.run.smithi035.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T15:13:50.273 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T15:13:50.467 INFO:teuthology.orchestra.run.smithi035.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T15:13:51.094 INFO:teuthology.orchestra.run.smithi035.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T15:13:51.095 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/138 2022-11-27T15:13:51.095 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/138 2022-11-27T15:13:51.095 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/138 2022-11-27T15:13:51.096 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/138 2022-11-27T15:13:51.096 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/138 2022-11-27T15:13:51.096 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/138 2022-11-27T15:13:51.096 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T15:13:51.096 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T15:13:51.097 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T15:13:51.097 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T15:13:51.097 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/138 2022-11-27T15:13:51.097 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/138 2022-11-27T15:13:51.098 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/138 2022-11-27T15:13:51.098 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/138 2022-11-27T15:13:51.099 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T15:13:51.099 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/138 2022-11-27T15:13:51.100 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/138 2022-11-27T15:13:51.100 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/138 2022-11-27T15:13:51.100 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/138 2022-11-27T15:13:51.101 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/138 2022-11-27T15:13:51.101 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/138 2022-11-27T15:13:51.102 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/138 2022-11-27T15:13:51.102 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/138 2022-11-27T15:13:51.103 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/138 2022-11-27T15:13:51.103 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/138 2022-11-27T15:13:51.104 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/138 2022-11-27T15:13:51.104 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/138 2022-11-27T15:13:51.104 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/138 2022-11-27T15:13:51.105 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/138 2022-11-27T15:13:51.105 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/138 2022-11-27T15:13:51.106 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/138 2022-11-27T15:13:51.106 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/138 2022-11-27T15:13:51.106 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/138 2022-11-27T15:13:51.107 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/138 2022-11-27T15:13:51.107 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/138 2022-11-27T15:13:51.108 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/138 2022-11-27T15:13:51.108 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/138 2022-11-27T15:13:51.108 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/138 2022-11-27T15:13:51.109 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-11-27T15:13:51.109 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-11-27T15:13:51.109 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-11-27T15:13:51.109 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-11-27T15:13:51.110 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-11-27T15:13:51.110 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-11-27T15:13:51.110 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-11-27T15:13:51.111 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-11-27T15:13:51.111 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-11-27T15:13:51.111 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-11-27T15:13:51.111 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-11-27T15:13:51.112 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-11-27T15:13:51.112 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-11-27T15:13:51.112 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-11-27T15:13:51.113 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-11-27T15:13:51.113 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-11-27T15:13:51.113 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-11-27T15:13:51.113 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-11-27T15:13:51.114 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-11-27T15:13:51.114 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-11-27T15:13:51.114 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-11-27T15:13:51.114 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-11-27T15:13:51.115 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-11-27T15:13:51.115 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-11-27T15:13:51.115 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-11-27T15:13:51.116 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-11-27T15:13:51.116 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-11-27T15:13:51.116 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-11-27T15:13:51.117 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-11-27T15:13:51.117 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-11-27T15:13:51.117 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T15:13:51.118 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-11-27T15:13:51.118 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-11-27T15:13:51.118 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 72/138 2022-11-27T15:13:51.119 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 73/138 2022-11-27T15:13:51.119 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 74/138 2022-11-27T15:13:51.119 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/138 2022-11-27T15:13:51.119 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 76/138 2022-11-27T15:13:51.120 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 77/138 2022-11-27T15:13:51.120 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 78/138 2022-11-27T15:13:51.121 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 79/138 2022-11-27T15:13:51.121 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 80/138 2022-11-27T15:13:51.121 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 81/138 2022-11-27T15:13:51.121 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/138 2022-11-27T15:13:51.122 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 83/138 2022-11-27T15:13:51.122 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 84/138 2022-11-27T15:13:51.122 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/138 2022-11-27T15:13:51.122 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 86/138 2022-11-27T15:13:51.123 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/138 2022-11-27T15:13:51.123 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 88/138 2022-11-27T15:13:51.123 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 89/138 2022-11-27T15:13:51.123 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 90/138 2022-11-27T15:13:51.124 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 91/138 2022-11-27T15:13:51.124 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 92/138 2022-11-27T15:13:51.124 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 93/138 2022-11-27T15:13:51.124 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 94/138 2022-11-27T15:13:51.125 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 95/138 2022-11-27T15:13:51.125 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/138 2022-11-27T15:13:51.125 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/138 2022-11-27T15:13:51.126 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 98/138 2022-11-27T15:13:51.126 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : bison-3.0.4-10.el8.x86_64 99/138 2022-11-27T15:13:51.126 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 100/138 2022-11-27T15:13:51.127 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 101/138 2022-11-27T15:13:51.127 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 102/138 2022-11-27T15:13:51.127 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 103/138 2022-11-27T15:13:51.127 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/138 2022-11-27T15:13:51.128 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 105/138 2022-11-27T15:13:51.128 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 106/138 2022-11-27T15:13:51.128 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 107/138 2022-11-27T15:13:51.128 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/138 2022-11-27T15:13:51.129 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 109/138 2022-11-27T15:13:51.129 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 110/138 2022-11-27T15:13:51.129 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 111/138 2022-11-27T15:13:51.129 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/138 2022-11-27T15:13:51.130 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/138 2022-11-27T15:13:51.130 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 114/138 2022-11-27T15:13:51.130 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : flex-2.6.1-9.el8.x86_64 115/138 2022-11-27T15:13:51.130 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 116/138 2022-11-27T15:13:51.131 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : jq-1.6-3.el8.x86_64 117/138 2022-11-27T15:13:51.131 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 118/138 2022-11-27T15:13:51.131 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : automake-1.16.1-7.el8.noarch 119/138 2022-11-27T15:13:51.131 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 120/138 2022-11-27T15:13:51.132 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 121/138 2022-11-27T15:13:51.132 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 122/138 2022-11-27T15:13:51.132 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/138 2022-11-27T15:13:51.132 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 124/138 2022-11-27T15:13:51.133 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 125/138 2022-11-27T15:13:51.133 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 126/138 2022-11-27T15:13:51.133 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 127/138 2022-11-27T15:13:51.133 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 128/138 2022-11-27T15:13:51.133 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 129/138 2022-11-27T15:13:51.134 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-11-27T15:13:51.134 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 131/138 2022-11-27T15:13:51.134 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-11-27T15:13:51.134 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-11-27T15:13:51.135 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T15:13:51.135 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-11-27T15:13:51.135 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-11-27T15:13:51.135 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-11-27T15:13:53.187 INFO:teuthology.orchestra.run.smithi035.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T15:13:53.188 INFO:teuthology.orchestra.run.smithi035.stdout:Installed products updated. 2022-11-27T15:13:53.449 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:53.449 INFO:teuthology.orchestra.run.smithi035.stdout:Upgraded: 2022-11-27T15:13:53.450 INFO:teuthology.orchestra.run.smithi035.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-11-27T15:13:53.450 INFO:teuthology.orchestra.run.smithi035.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.451 INFO:teuthology.orchestra.run.smithi035.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.451 INFO:teuthology.orchestra.run.smithi035.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-11-27T15:13:53.452 INFO:teuthology.orchestra.run.smithi035.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-11-27T15:13:53.452 INFO:teuthology.orchestra.run.smithi035.stdout:Installed: 2022-11-27T15:13:53.453 INFO:teuthology.orchestra.run.smithi035.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-11-27T15:13:53.453 INFO:teuthology.orchestra.run.smithi035.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T15:13:53.453 INFO:teuthology.orchestra.run.smithi035.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T15:13:53.454 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.454 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.455 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.456 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.457 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:53.458 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.458 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.458 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.458 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:53.459 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:53.459 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:53.459 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:53.459 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:53.460 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.460 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.460 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:53.460 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.461 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.461 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.461 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:53.461 INFO:teuthology.orchestra.run.smithi035.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:53.462 INFO:teuthology.orchestra.run.smithi035.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.462 INFO:teuthology.orchestra.run.smithi035.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T15:13:53.462 INFO:teuthology.orchestra.run.smithi035.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-11-27T15:13:53.463 INFO:teuthology.orchestra.run.smithi035.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T15:13:53.463 INFO:teuthology.orchestra.run.smithi035.stdout: fuse-2.9.7-15.el8.x86_64 2022-11-27T15:13:53.464 INFO:teuthology.orchestra.run.smithi035.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T15:13:53.464 INFO:teuthology.orchestra.run.smithi035.stdout: jq-1.6-3.el8.x86_64 2022-11-27T15:13:53.465 INFO:teuthology.orchestra.run.smithi035.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T15:13:53.465 INFO:teuthology.orchestra.run.smithi035.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-11-27T15:13:53.466 INFO:teuthology.orchestra.run.smithi035.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T15:13:53.466 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.467 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.467 INFO:teuthology.orchestra.run.smithi035.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.468 INFO:teuthology.orchestra.run.smithi035.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-11-27T15:13:53.468 INFO:teuthology.orchestra.run.smithi035.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-11-27T15:13:53.468 INFO:teuthology.orchestra.run.smithi035.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T15:13:53.469 INFO:teuthology.orchestra.run.smithi035.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T15:13:53.469 INFO:teuthology.orchestra.run.smithi035.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-11-27T15:13:53.469 INFO:teuthology.orchestra.run.smithi035.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T15:13:53.469 INFO:teuthology.orchestra.run.smithi035.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.470 INFO:teuthology.orchestra.run.smithi035.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.470 INFO:teuthology.orchestra.run.smithi035.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T15:13:53.470 INFO:teuthology.orchestra.run.smithi035.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.471 INFO:teuthology.orchestra.run.smithi035.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-11-27T15:13:53.472 INFO:teuthology.orchestra.run.smithi035.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T15:13:53.472 INFO:teuthology.orchestra.run.smithi035.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T15:13:53.472 INFO:teuthology.orchestra.run.smithi035.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T15:13:53.472 INFO:teuthology.orchestra.run.smithi035.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-11-27T15:13:53.473 INFO:teuthology.orchestra.run.smithi035.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-11-27T15:13:53.473 INFO:teuthology.orchestra.run.smithi035.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T15:13:53.473 INFO:teuthology.orchestra.run.smithi035.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T15:13:53.473 INFO:teuthology.orchestra.run.smithi035.stdout: openblas-0.3.15-3.el8.x86_64 2022-11-27T15:13:53.474 INFO:teuthology.orchestra.run.smithi035.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-11-27T15:13:53.474 INFO:teuthology.orchestra.run.smithi035.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-11-27T15:13:53.475 INFO:teuthology.orchestra.run.smithi035.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-11-27T15:13:53.475 INFO:teuthology.orchestra.run.smithi035.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-11-27T15:13:53.475 INFO:teuthology.orchestra.run.smithi035.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-11-27T15:13:53.475 INFO:teuthology.orchestra.run.smithi035.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T15:13:53.476 INFO:teuthology.orchestra.run.smithi035.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T15:13:53.476 INFO:teuthology.orchestra.run.smithi035.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T15:13:53.477 INFO:teuthology.orchestra.run.smithi035.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T15:13:53.477 INFO:teuthology.orchestra.run.smithi035.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T15:13:53.478 INFO:teuthology.orchestra.run.smithi035.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T15:13:53.478 INFO:teuthology.orchestra.run.smithi035.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T15:13:53.478 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T15:13:53.478 INFO:teuthology.orchestra.run.smithi035.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.479 INFO:teuthology.orchestra.run.smithi035.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.479 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.479 INFO:teuthology.orchestra.run.smithi035.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T15:13:53.479 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T15:13:53.480 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T15:13:53.480 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T15:13:53.480 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T15:13:53.480 INFO:teuthology.orchestra.run.smithi035.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T15:13:53.480 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T15:13:53.481 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T15:13:53.481 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T15:13:53.481 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T15:13:53.481 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T15:13:53.482 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T15:13:53.482 INFO:teuthology.orchestra.run.smithi035.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T15:13:53.482 INFO:teuthology.orchestra.run.smithi035.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T15:13:53.483 INFO:teuthology.orchestra.run.smithi035.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T15:13:53.483 INFO:teuthology.orchestra.run.smithi035.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T15:13:53.483 INFO:teuthology.orchestra.run.smithi035.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T15:13:53.484 INFO:teuthology.orchestra.run.smithi035.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T15:13:53.484 INFO:teuthology.orchestra.run.smithi035.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T15:13:53.485 INFO:teuthology.orchestra.run.smithi035.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T15:13:53.485 INFO:teuthology.orchestra.run.smithi035.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T15:13:53.486 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T15:13:53.486 INFO:teuthology.orchestra.run.smithi035.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T15:13:53.487 INFO:teuthology.orchestra.run.smithi035.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T15:13:53.487 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T15:13:53.488 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T15:13:53.488 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T15:13:53.488 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T15:13:53.489 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T15:13:53.489 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.490 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.490 INFO:teuthology.orchestra.run.smithi035.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T15:13:53.491 INFO:teuthology.orchestra.run.smithi035.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T15:13:53.491 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.492 INFO:teuthology.orchestra.run.smithi035.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T15:13:53.492 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T15:13:53.492 INFO:teuthology.orchestra.run.smithi035.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-11-27T15:13:53.493 INFO:teuthology.orchestra.run.smithi035.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T15:13:53.493 INFO:teuthology.orchestra.run.smithi035.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T15:13:53.494 INFO:teuthology.orchestra.run.smithi035.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T15:13:53.494 INFO:teuthology.orchestra.run.smithi035.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T15:13:53.494 INFO:teuthology.orchestra.run.smithi035.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T15:13:53.495 INFO:teuthology.orchestra.run.smithi035.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T15:13:53.495 INFO:teuthology.orchestra.run.smithi035.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T15:13:53.495 INFO:teuthology.orchestra.run.smithi035.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T15:13:53.496 INFO:teuthology.orchestra.run.smithi035.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T15:13:53.496 INFO:teuthology.orchestra.run.smithi035.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T15:13:53.496 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.496 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.497 INFO:teuthology.orchestra.run.smithi035.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T15:13:53.497 INFO:teuthology.orchestra.run.smithi035.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T15:13:53.497 INFO:teuthology.orchestra.run.smithi035.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T15:13:53.497 INFO:teuthology.orchestra.run.smithi035.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T15:13:53.498 INFO:teuthology.orchestra.run.smithi035.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T15:13:53.498 INFO:teuthology.orchestra.run.smithi035.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T15:13:53.499 INFO:teuthology.orchestra.run.smithi035.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-11-27T15:13:53.499 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:53.499 INFO:teuthology.orchestra.run.smithi035.stdout:Complete! 2022-11-27T15:13:53.626 DEBUG:teuthology.parallel:result is None 2022-11-27T15:13:53.627 DEBUG:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2022-11-27T15:13:53.686 INFO:teuthology.orchestra.run.smithi035.stdout:RedHatEnterprise 2022-11-27T15:13:53.688 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T15:13:53.689 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T15:13:53.850 DEBUG:teuthology.orchestra.run.smithi035:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T15:13:53.874 INFO:teuthology.orchestra.run.smithi035.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T15:13:53.876 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T15:13:53.876 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T15:13:53.877 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2022-11-27T15:13:53.943 INFO:teuthology.orchestra.run.smithi085.stdout:RedHatEnterprise 2022-11-27T15:13:53.944 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T15:13:53.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T15:13:54.096 DEBUG:teuthology.orchestra.run.smithi085:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T15:13:54.121 INFO:teuthology.orchestra.run.smithi085.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T15:13:54.122 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T15:13:54.122 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T15:13:54.123 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-11-27T15:13:54.124 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:13:54.124 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T15:13:54.153 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:13:54.154 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T15:13:54.191 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-11-27T15:13:54.191 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:13:54.192 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T15:13:54.222 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T15:13:54.291 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:13:54.292 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T15:13:54.322 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T15:13:54.390 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-11-27T15:13:54.390 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:13:54.391 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T15:13:54.419 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T15:13:54.486 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:13:54.487 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T15:13:54.515 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T15:13:54.582 INFO:teuthology.run_tasks:Running task ceph... 2022-11-27T15:13:54.648 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-11-27T15:13:54.648 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 777 /var/log/ceph 2022-11-27T15:13:54.651 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod 777 /var/log/ceph 2022-11-27T15:13:54.678 INFO:tasks.ceph:Disabling ceph logrotate... 2022-11-27T15:13:54.678 DEBUG:teuthology.orchestra.run.smithi035:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T15:13:54.720 DEBUG:teuthology.orchestra.run.smithi085:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T15:13:54.747 INFO:tasks.ceph:Creating extra log directories... 2022-11-27T15:13:54.748 DEBUG:teuthology.orchestra.run.smithi035:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T15:13:54.788 DEBUG:teuthology.orchestra.run.smithi085:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T15:13:54.825 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-11-27T15:13:54.825 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-11-27T15:13:54.826 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-11-27T15:13:54.826 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:13:54.826 DEBUG:teuthology.orchestra.run.smithi035:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T15:13:54.870 DEBUG:teuthology.orchestra.run.smithi035:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T15:13:54.988 DEBUG:teuthology.orchestra.run.smithi035:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T15:13:55.016 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:13:55.016 DEBUG:teuthology.orchestra.run.smithi085:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T15:13:55.033 DEBUG:teuthology.orchestra.run.smithi085:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T15:13:55.159 DEBUG:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T15:13:55.205 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-11-27T15:13:55.206 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-11-27T15:13:55.206 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088602', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '7088602', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi035.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOFj8mbVZguKfonD5ijtA/JHmS+dW9gjdQiLWKApgDnyPZ7ExrXC4MKJFBpoKDqeL64+yGbw5vxUIDCsIstSCXhzXoojsVeRibEQZu7hJidMK/k6Ay8PDamRs3NYHlXgl7vsAST9F9qvdbIcbyzUzjKYcNGjRQ/WbrSsye46t1HSJuNvkcW0+P8U5W1wlNqL6i4iI3qU8ax7l+hKqCrUzEBK4MLa/OUSZancBHi5wpF3Un9SH+bcyAT3pmEUkSmOqTQSipDpCiL7yWLda6PSv0tH2WJc7Ombv+iWgVxbZ72vmX187DX9hF5i13ljzI+nJzkzWMyA/4o4PQYcQ7TphCi4WUOd+8KzToAmGXgiXtMx/bcJClOoL1kwY7P9u+vPC3ChUvyClvOliCvIjbdl3q/HrZ/i41PkBrIiUXU6U9hE79hcCi33gd95/Jiy3BHjLfI3v9hpdFdURAxxEpNvsFm/8x75FSpWR2oq+lv5oJFDvGVR+3+pYWXDAhKjd1U20=', 'smithi085.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+owsi1HBloX+kA4jlHQiMLnh5+1NeWWDubHWHp0KFXTwr9qTixFX2ps2QQ+6yset+FnELQh7Rh3afVLA6BnoKv8FrNq1UbQ+19/RNZaAIGdCilObbYbtCB5P90EuNnv/ZXc5R7qZXEDeTZKLQ70JChdbW6Agj+rpNYUDPm7aOOoT6WzW6BZJv2Xq7VpVH0tCq+G7l+7uQgSr0TOPsgpWEh7MssLJy/WrsRJTogCPRd2y6Onlw5gFKbTuul/13Usc0NqBzHok08MjoLdzW81L7qbDjVWL+LAf0Abg0mLvKywQIJrUslgiHkrQRaknQ6T3ntRwaaQDMqcM1UE0rC0leFFSMbVGGexKW3eNdGGkv8NsRzajmLtF62DO2Cjk+ZdP1awEuTWh00+TiildRt0R8nKmKdPTUZH3PqErgOlDImQLKmix/8B8RBP21gta3up6QdMXjEVUbbeEwk0wLsIyKyF65SKvAKqPw4cQvuUBAWmBdnH8/E8AphxbtR42J5Kc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_recovery_pool']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-23_17:24:26', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23126'} 2022-11-27T15:13:55.207 DEBUG:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T15:13:55.210 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T15:13:55.253 DEBUG:teuthology.orchestra.run.smithi035:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T15:13:55.268 DEBUG:teuthology.orchestra.run.smithi085:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T15:13:55.321 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:13:55.322 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T15:13:55.351 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T15:13:55.351 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_1 2022-11-27T15:13:55.409 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T15:13:55.410 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:13:55.410 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 367182 Links: 1 2022-11-27T15:13:55.410 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:13:55.410 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:13:55.411 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2022-11-27 15:13:50.747084368 +0000 2022-11-27T15:13:55.411 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2022-11-27 15:10:56.763996380 +0000 2022-11-27T15:13:55.411 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2022-11-27 15:10:56.763996380 +0000 2022-11-27T15:13:55.411 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2022-11-27T15:13:55.412 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T15:13:55.477 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2022-11-27T15:13:55.477 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2022-11-27T15:13:55.477 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000250747 s, 2.0 MB/s 2022-11-27T15:13:55.479 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T15:13:55.537 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_2 2022-11-27T15:13:55.596 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T15:13:55.596 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:13:55.597 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 367296 Links: 1 2022-11-27T15:13:55.597 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:13:55.597 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:13:55.598 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2022-11-27 15:13:50.747084368 +0000 2022-11-27T15:13:55.598 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2022-11-27 15:10:57.337984494 +0000 2022-11-27T15:13:55.598 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2022-11-27 15:10:57.337984494 +0000 2022-11-27T15:13:55.599 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2022-11-27T15:13:55.600 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T15:13:55.665 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2022-11-27T15:13:55.666 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2022-11-27T15:13:55.666 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000211167 s, 2.4 MB/s 2022-11-27T15:13:55.667 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T15:13:55.724 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_3 2022-11-27T15:13:55.782 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T15:13:55.782 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:13:55.783 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 368902 Links: 1 2022-11-27T15:13:55.783 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:13:55.784 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:13:55.784 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2022-11-27 15:13:50.748084348 +0000 2022-11-27T15:13:55.785 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2022-11-27 15:10:57.897972899 +0000 2022-11-27T15:13:55.785 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2022-11-27 15:10:57.897972899 +0000 2022-11-27T15:13:55.786 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2022-11-27T15:13:55.786 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T15:13:55.849 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2022-11-27T15:13:55.849 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2022-11-27T15:13:55.849 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000206323 s, 2.5 MB/s 2022-11-27T15:13:55.850 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T15:13:55.909 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_4 2022-11-27T15:13:55.966 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T15:13:55.966 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:13:55.967 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 364050 Links: 1 2022-11-27T15:13:55.967 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:13:55.967 INFO:teuthology.orchestra.run.smithi035.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:13:55.968 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2022-11-27 15:13:50.748084348 +0000 2022-11-27T15:13:55.968 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2022-11-27 15:10:58.509960226 +0000 2022-11-27T15:13:55.968 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2022-11-27 15:10:58.509960226 +0000 2022-11-27T15:13:55.969 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2022-11-27T15:13:55.969 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T15:13:56.032 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2022-11-27T15:13:56.033 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2022-11-27T15:13:56.033 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000204847 s, 2.5 MB/s 2022-11-27T15:13:56.034 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T15:13:56.097 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T15:13:56.116 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:13:56.117 DEBUG:teuthology.orchestra.run.smithi085:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T15:13:56.141 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T15:13:56.141 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_1 2022-11-27T15:13:56.200 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T15:13:56.200 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:13:56.201 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 365215 Links: 1 2022-11-27T15:13:56.201 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:13:56.202 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:13:56.202 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2022-11-27 15:13:39.381929545 +0000 2022-11-27T15:13:56.203 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2022-11-27 15:10:43.420474839 +0000 2022-11-27T15:13:56.203 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2022-11-27 15:10:43.420474839 +0000 2022-11-27T15:13:56.204 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2022-11-27T15:13:56.204 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T15:13:56.267 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2022-11-27T15:13:56.268 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2022-11-27T15:13:56.268 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.00024299 s, 2.1 MB/s 2022-11-27T15:13:56.269 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T15:13:56.330 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_2 2022-11-27T15:13:56.388 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T15:13:56.388 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:13:56.388 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 364540 Links: 1 2022-11-27T15:13:56.389 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:13:56.389 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:13:56.389 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2022-11-27 15:13:39.381929545 +0000 2022-11-27T15:13:56.389 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2022-11-27 15:10:43.976465322 +0000 2022-11-27T15:13:56.390 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2022-11-27 15:10:43.976465322 +0000 2022-11-27T15:13:56.390 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2022-11-27T15:13:56.391 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T15:13:56.453 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2022-11-27T15:13:56.454 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2022-11-27T15:13:56.454 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000233358 s, 2.2 MB/s 2022-11-27T15:13:56.455 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T15:13:56.515 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_3 2022-11-27T15:13:56.572 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T15:13:56.573 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:13:56.573 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 365356 Links: 1 2022-11-27T15:13:56.573 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:13:56.574 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:13:56.574 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2022-11-27 15:13:39.381929545 +0000 2022-11-27T15:13:56.574 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2022-11-27 15:10:44.528455873 +0000 2022-11-27T15:13:56.574 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2022-11-27 15:10:44.528455873 +0000 2022-11-27T15:13:56.575 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2022-11-27T15:13:56.575 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T15:13:56.638 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2022-11-27T15:13:56.638 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2022-11-27T15:13:56.639 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000193852 s, 2.6 MB/s 2022-11-27T15:13:56.640 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T15:13:56.698 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_4 2022-11-27T15:13:56.755 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T15:13:56.756 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:13:56.756 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 363421 Links: 1 2022-11-27T15:13:56.757 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:13:56.757 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:13:56.757 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2022-11-27 15:13:39.381929545 +0000 2022-11-27T15:13:56.757 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2022-11-27 15:10:45.084446356 +0000 2022-11-27T15:13:56.758 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2022-11-27 15:10:45.084446356 +0000 2022-11-27T15:13:56.758 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2022-11-27T15:13:56.758 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T15:13:56.821 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2022-11-27T15:13:56.822 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2022-11-27T15:13:56.822 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000217577 s, 2.4 MB/s 2022-11-27T15:13:56.823 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T15:13:56.881 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T15:13:56.881 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi035.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi085.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T15:13:56.882 INFO:tasks.ceph:Generating config... 2022-11-27T15:13:56.884 INFO:tasks.ceph:[client] client mount timeout = 600 2022-11-27T15:13:56.884 INFO:tasks.ceph:[client] debug client = 20 2022-11-27T15:13:56.884 INFO:tasks.ceph:[client] debug ms = 1 2022-11-27T15:13:56.885 INFO:tasks.ceph:[client] pid file = 2022-11-27T15:13:56.885 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-11-27T15:13:56.885 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-11-27T15:13:56.886 INFO:tasks.ceph:[global] ms die on skipped message = False 2022-11-27T15:13:56.886 INFO:tasks.ceph:[mds] debug mds = 20 2022-11-27T15:13:56.886 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-11-27T15:13:56.887 INFO:tasks.ceph:[mds] debug ms = 1 2022-11-27T15:13:56.887 INFO:tasks.ceph:[mds] mds debug frag = True 2022-11-27T15:13:56.888 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-11-27T15:13:56.888 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-11-27T15:13:56.888 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-11-27T15:13:56.888 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-11-27T15:13:56.889 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-11-27T15:13:56.889 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-11-27T15:13:56.890 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-11-27T15:13:56.890 INFO:tasks.ceph:[mgr] debug ms = 1 2022-11-27T15:13:56.891 INFO:tasks.ceph:[mon] debug mon = 20 2022-11-27T15:13:56.892 INFO:tasks.ceph:[mon] debug ms = 1 2022-11-27T15:13:56.892 INFO:tasks.ceph:[mon] debug paxos = 20 2022-11-27T15:13:56.893 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-11-27T15:13:56.893 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-11-27T15:13:56.894 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-11-27T15:13:56.894 INFO:tasks.ceph:[osd] debug bluefs = 20 2022-11-27T15:13:56.895 INFO:tasks.ceph:[osd] debug bluestore = 20 2022-11-27T15:13:56.895 INFO:tasks.ceph:[osd] debug ms = 1 2022-11-27T15:13:56.896 INFO:tasks.ceph:[osd] debug osd = 20 2022-11-27T15:13:56.896 INFO:tasks.ceph:[osd] debug rocksdb = 10 2022-11-27T15:13:56.897 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-11-27T15:13:56.897 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-11-27T15:13:56.898 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-11-27T15:13:56.898 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-11-27T15:13:56.898 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-11-27T15:13:56.899 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-11-27T15:13:56.899 INFO:tasks.ceph:Setting up mon.a... 2022-11-27T15:13:56.900 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-11-27T15:13:56.989 INFO:teuthology.orchestra.run.smithi035.stdout:creating /etc/ceph/ceph.keyring 2022-11-27T15:13:56.993 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-11-27T15:13:57.043 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T15:13:57.072 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.35'), ('mon.b', '172.21.15.85'), ('mon.c', '[v2:172.21.15.85:3301,v1:172.21.15.85:6790]')] 2022-11-27T15:13:57.073 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.35,172.21.15.85,[v2:172.21.15.85:3301,v1:172.21.15.85:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-11-27T15:13:57.074 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:13:57.074 DEBUG:teuthology.orchestra.run.smithi035:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T15:13:57.130 DEBUG:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.35 --add b 172.21.15.85 --addv c '[v2:172.21.15.85:3301,v1:172.21.15.85:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:13:57.220 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:13:57.220 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: generated fsid 1a1c2d0e-bda9-4965-9029-a4d82eabd38a 2022-11-27T15:13:57.221 INFO:teuthology.orchestra.run.smithi035.stdout:setting min_mon_release = pacific 2022-11-27T15:13:57.221 INFO:teuthology.orchestra.run.smithi035.stdout:epoch 0 2022-11-27T15:13:57.221 INFO:teuthology.orchestra.run.smithi035.stdout:fsid 1a1c2d0e-bda9-4965-9029-a4d82eabd38a 2022-11-27T15:13:57.222 INFO:teuthology.orchestra.run.smithi035.stdout:last_changed 2022-11-27T15:13:57.219852+0000 2022-11-27T15:13:57.222 INFO:teuthology.orchestra.run.smithi035.stdout:created 2022-11-27T15:13:57.219852+0000 2022-11-27T15:13:57.224 INFO:teuthology.orchestra.run.smithi035.stdout:min_mon_release 16 (pacific) 2022-11-27T15:13:57.224 INFO:teuthology.orchestra.run.smithi035.stdout:election_strategy: 1 2022-11-27T15:13:57.225 INFO:teuthology.orchestra.run.smithi035.stdout:0: [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] mon.a 2022-11-27T15:13:57.225 INFO:teuthology.orchestra.run.smithi035.stdout:1: [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] mon.b 2022-11-27T15:13:57.225 INFO:teuthology.orchestra.run.smithi035.stdout:2: [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] mon.c 2022-11-27T15:13:57.226 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-11-27T15:13:57.228 DEBUG:teuthology.orchestra.run.smithi035:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T15:13:57.247 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1a1c2d0e-bda9-4965-9029-a4d82eabd38a... 2022-11-27T15:13:57.248 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T15:13:57.290 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T15:13:57.335 INFO:teuthology.orchestra.run.smithi035.stdout:[global] 2022-11-27T15:13:57.335 INFO:teuthology.orchestra.run.smithi035.stdout: chdir = "" 2022-11-27T15:13:57.336 INFO:teuthology.orchestra.run.smithi035.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T15:13:57.336 INFO:teuthology.orchestra.run.smithi035.stdout: auth supported = cephx 2022-11-27T15:13:57.337 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.337 INFO:teuthology.orchestra.run.smithi035.stdout: filestore xattr use omap = true 2022-11-27T15:13:57.338 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.338 INFO:teuthology.orchestra.run.smithi035.stdout: mon clock drift allowed = 1.000 2022-11-27T15:13:57.339 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.339 INFO:teuthology.orchestra.run.smithi035.stdout: osd crush chooseleaf type = 0 2022-11-27T15:13:57.340 INFO:teuthology.orchestra.run.smithi035.stdout: auth debug = true 2022-11-27T15:13:57.340 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.341 INFO:teuthology.orchestra.run.smithi035.stdout: ms die on old message = true 2022-11-27T15:13:57.341 INFO:teuthology.orchestra.run.smithi035.stdout: ms die on bug = true 2022-11-27T15:13:57.342 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.342 INFO:teuthology.orchestra.run.smithi035.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T15:13:57.342 INFO:teuthology.orchestra.run.smithi035.stdout: mon pg warn max object skew = 0 2022-11-27T15:13:57.343 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.343 INFO:teuthology.orchestra.run.smithi035.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T15:13:57.344 INFO:teuthology.orchestra.run.smithi035.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T15:13:57.344 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.345 INFO:teuthology.orchestra.run.smithi035.stdout: osd pool default size = 2 2022-11-27T15:13:57.345 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.346 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd allow primary affinity = true 2022-11-27T15:13:57.346 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd allow pg remap = true 2022-11-27T15:13:57.347 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on legacy crush tunables = false 2022-11-27T15:13:57.347 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on crush straw calc version zero = false 2022-11-27T15:13:57.348 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on no sortbitwise = false 2022-11-27T15:13:57.348 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on osd down out interval zero = false 2022-11-27T15:13:57.349 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on too few osds = false 2022-11-27T15:13:57.349 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T15:13:57.349 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T15:13:57.350 INFO:teuthology.orchestra.run.smithi035.stdout: mon_allow_pool_size_one = true 2022-11-27T15:13:57.351 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.351 INFO:teuthology.orchestra.run.smithi035.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T15:13:57.352 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.352 INFO:teuthology.orchestra.run.smithi035.stdout: osd default data pool replay window = 5 2022-11-27T15:13:57.352 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.352 INFO:teuthology.orchestra.run.smithi035.stdout: mon allow pool delete = true 2022-11-27T15:13:57.353 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.353 INFO:teuthology.orchestra.run.smithi035.stdout: mon cluster log file level = debug 2022-11-27T15:13:57.353 INFO:teuthology.orchestra.run.smithi035.stdout: debug asserts on shutdown = true 2022-11-27T15:13:57.353 INFO:teuthology.orchestra.run.smithi035.stdout: mon health detail to clog = false 2022-11-27T15:13:57.354 INFO:teuthology.orchestra.run.smithi035.stdout: mon host = "172.21.15.35,172.21.15.85,[v2:172.21.15.85:3301,v1:172.21.15.85:6790]" 2022-11-27T15:13:57.354 INFO:teuthology.orchestra.run.smithi035.stdout: ms die on skipped message = False 2022-11-27T15:13:57.354 INFO:teuthology.orchestra.run.smithi035.stdout: fsid = 1a1c2d0e-bda9-4965-9029-a4d82eabd38a 2022-11-27T15:13:57.354 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.355 INFO:teuthology.orchestra.run.smithi035.stdout:[osd] 2022-11-27T15:13:57.355 INFO:teuthology.orchestra.run.smithi035.stdout: osd journal size = 100 2022-11-27T15:13:57.355 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.355 INFO:teuthology.orchestra.run.smithi035.stdout: osd scrub load threshold = 5.0 2022-11-27T15:13:57.356 INFO:teuthology.orchestra.run.smithi035.stdout: osd scrub max interval = 600 2022-11-27T15:13:57.356 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.356 INFO:teuthology.orchestra.run.smithi035.stdout: osd recover clone overlap = true 2022-11-27T15:13:57.356 INFO:teuthology.orchestra.run.smithi035.stdout: osd recovery max chunk = 1048576 2022-11-27T15:13:57.356 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.357 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug shutdown = true 2022-11-27T15:13:57.357 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug op order = true 2022-11-27T15:13:57.357 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug verify stray on activate = true 2022-11-27T15:13:57.357 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.358 INFO:teuthology.orchestra.run.smithi035.stdout: osd open classes on start = true 2022-11-27T15:13:57.358 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug pg log writeout = true 2022-11-27T15:13:57.358 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.358 INFO:teuthology.orchestra.run.smithi035.stdout: osd deep scrub update digest min age = 30 2022-11-27T15:13:57.359 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.359 INFO:teuthology.orchestra.run.smithi035.stdout: osd map max advance = 10 2022-11-27T15:13:57.359 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.360 INFO:teuthology.orchestra.run.smithi035.stdout: journal zero on create = true 2022-11-27T15:13:57.360 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.360 INFO:teuthology.orchestra.run.smithi035.stdout: filestore ondisk finisher threads = 3 2022-11-27T15:13:57.360 INFO:teuthology.orchestra.run.smithi035.stdout: filestore apply finisher threads = 3 2022-11-27T15:13:57.361 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.361 INFO:teuthology.orchestra.run.smithi035.stdout: bdev debug aio = true 2022-11-27T15:13:57.361 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug misdirected ops = true 2022-11-27T15:13:57.361 INFO:teuthology.orchestra.run.smithi035.stdout: bluestore block size = 96636764160 2022-11-27T15:13:57.362 INFO:teuthology.orchestra.run.smithi035.stdout: bluestore fsck on mount = True 2022-11-27T15:13:57.362 INFO:teuthology.orchestra.run.smithi035.stdout: debug bluefs = 20 2022-11-27T15:13:57.362 INFO:teuthology.orchestra.run.smithi035.stdout: debug bluestore = 20 2022-11-27T15:13:57.362 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2022-11-27T15:13:57.363 INFO:teuthology.orchestra.run.smithi035.stdout: debug osd = 20 2022-11-27T15:13:57.363 INFO:teuthology.orchestra.run.smithi035.stdout: debug rocksdb = 10 2022-11-27T15:13:57.363 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T15:13:57.364 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd full ratio = 0.9 2022-11-27T15:13:57.364 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd nearfull ratio = 0.8 2022-11-27T15:13:57.364 INFO:teuthology.orchestra.run.smithi035.stdout: osd failsafe full ratio = 0.95 2022-11-27T15:13:57.364 INFO:teuthology.orchestra.run.smithi035.stdout: osd objectstore = bluestore 2022-11-27T15:13:57.365 INFO:teuthology.orchestra.run.smithi035.stdout: osd op complaint time = 180 2022-11-27T15:13:57.365 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.365 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr] 2022-11-27T15:13:57.365 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2022-11-27T15:13:57.366 INFO:teuthology.orchestra.run.smithi035.stdout: debug mgr = 20 2022-11-27T15:13:57.366 INFO:teuthology.orchestra.run.smithi035.stdout: debug mon = 20 2022-11-27T15:13:57.366 INFO:teuthology.orchestra.run.smithi035.stdout: debug auth = 20 2022-11-27T15:13:57.366 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min pgs per osd = 4 2022-11-27T15:13:57.367 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min bytes per osd = 10 2022-11-27T15:13:57.367 INFO:teuthology.orchestra.run.smithi035.stdout: mgr/telemetry/nag = false 2022-11-27T15:13:57.367 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.367 INFO:teuthology.orchestra.run.smithi035.stdout:[mon] 2022-11-27T15:13:57.368 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2022-11-27T15:13:57.368 INFO:teuthology.orchestra.run.smithi035.stdout: debug mon = 20 2022-11-27T15:13:57.368 INFO:teuthology.orchestra.run.smithi035.stdout: debug paxos = 20 2022-11-27T15:13:57.369 INFO:teuthology.orchestra.run.smithi035.stdout: debug auth = 20 2022-11-27T15:13:57.369 INFO:teuthology.orchestra.run.smithi035.stdout: mon data avail warn = 5 2022-11-27T15:13:57.369 INFO:teuthology.orchestra.run.smithi035.stdout: mon mgr mkfs grace = 240 2022-11-27T15:13:57.369 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min pgs per osd = 4 2022-11-27T15:13:57.370 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd reporter subtree level = osd 2022-11-27T15:13:57.370 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd prime pg temp = true 2022-11-27T15:13:57.370 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min bytes per osd = 10 2022-11-27T15:13:57.370 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.371 INFO:teuthology.orchestra.run.smithi035.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T15:13:57.371 INFO:teuthology.orchestra.run.smithi035.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T15:13:57.371 INFO:teuthology.orchestra.run.smithi035.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T15:13:57.371 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.372 INFO:teuthology.orchestra.run.smithi035.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T15:13:57.372 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T15:13:57.372 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T15:13:57.372 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.373 INFO:teuthology.orchestra.run.smithi035.stdout: # 1m isn't quite enough 2022-11-27T15:13:57.373 INFO:teuthology.orchestra.run.smithi035.stdout: mon_down_mkfs_grace = 2m 2022-11-27T15:13:57.373 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.373 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_filestore_osds = false 2022-11-27T15:13:57.374 INFO:teuthology.orchestra.run.smithi035.stdout: mon op complaint time = 120 2022-11-27T15:13:57.374 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:13:57.374 INFO:teuthology.orchestra.run.smithi035.stdout:[client] 2022-11-27T15:13:57.374 INFO:teuthology.orchestra.run.smithi035.stdout: rgw cache enabled = true 2022-11-27T15:13:57.375 INFO:teuthology.orchestra.run.smithi035.stdout: rgw enable ops log = true 2022-11-27T15:13:57.375 INFO:teuthology.orchestra.run.smithi035.stdout: rgw enable usage log = true 2022-11-27T15:13:57.375 INFO:teuthology.orchestra.run.smithi035.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T15:13:57.375 INFO:teuthology.orchestra.run.smithi035.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T15:13:57.376 INFO:teuthology.orchestra.run.smithi035.stdout: client mount timeout = 600 2022-11-27T15:13:57.376 INFO:teuthology.orchestra.run.smithi035.stdout: debug client = 20 2022-11-27T15:13:57.376 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2022-11-27T15:13:57.376 INFO:teuthology.orchestra.run.smithi035.stdout: pid file = "" 2022-11-27T15:13:57.377 INFO:teuthology.orchestra.run.smithi035.stdout: rados mon op timeout = 900 2022-11-27T15:13:57.377 INFO:teuthology.orchestra.run.smithi035.stdout: rados osd op timeout = 900 2022-11-27T15:13:57.377 INFO:teuthology.orchestra.run.smithi035.stdout:[mon.a] 2022-11-27T15:13:57.377 INFO:teuthology.orchestra.run.smithi035.stdout:[mon.b] 2022-11-27T15:13:57.378 INFO:teuthology.orchestra.run.smithi035.stdout:[mon.c] 2022-11-27T15:13:57.378 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.a] 2022-11-27T15:13:57.378 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.c] 2022-11-27T15:13:57.378 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.b] 2022-11-27T15:13:57.379 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.d] 2022-11-27T15:13:57.379 INFO:teuthology.orchestra.run.smithi035.stdout:[mds] 2022-11-27T15:13:57.379 INFO:teuthology.orchestra.run.smithi035.stdout: debug mds = 20 2022-11-27T15:13:57.379 INFO:teuthology.orchestra.run.smithi035.stdout: debug mds balancer = 20 2022-11-27T15:13:57.380 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2022-11-27T15:13:57.380 INFO:teuthology.orchestra.run.smithi035.stdout: mds debug frag = True 2022-11-27T15:13:57.380 INFO:teuthology.orchestra.run.smithi035.stdout: mds debug scatterstat = True 2022-11-27T15:13:57.380 INFO:teuthology.orchestra.run.smithi035.stdout: mds op complaint time = 180 2022-11-27T15:13:57.381 INFO:teuthology.orchestra.run.smithi035.stdout: mds verify scatter = True 2022-11-27T15:13:57.381 INFO:teuthology.orchestra.run.smithi035.stdout: osd op complaint time = 180 2022-11-27T15:13:57.381 INFO:teuthology.orchestra.run.smithi035.stdout: rados mon op timeout = 900 2022-11-27T15:13:57.381 INFO:teuthology.orchestra.run.smithi035.stdout: rados osd op timeout = 900 2022-11-27T15:13:57.384 INFO:teuthology.orchestra.run.smithi085.stdout:[global] 2022-11-27T15:13:57.384 INFO:teuthology.orchestra.run.smithi085.stdout: chdir = "" 2022-11-27T15:13:57.384 INFO:teuthology.orchestra.run.smithi085.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T15:13:57.384 INFO:teuthology.orchestra.run.smithi085.stdout: auth supported = cephx 2022-11-27T15:13:57.385 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.385 INFO:teuthology.orchestra.run.smithi085.stdout: filestore xattr use omap = true 2022-11-27T15:13:57.385 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.386 INFO:teuthology.orchestra.run.smithi085.stdout: mon clock drift allowed = 1.000 2022-11-27T15:13:57.386 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.386 INFO:teuthology.orchestra.run.smithi085.stdout: osd crush chooseleaf type = 0 2022-11-27T15:13:57.386 INFO:teuthology.orchestra.run.smithi085.stdout: auth debug = true 2022-11-27T15:13:57.387 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.387 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on old message = true 2022-11-27T15:13:57.387 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on bug = true 2022-11-27T15:13:57.387 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.388 INFO:teuthology.orchestra.run.smithi085.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T15:13:57.388 INFO:teuthology.orchestra.run.smithi085.stdout: mon pg warn max object skew = 0 2022-11-27T15:13:57.388 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.388 INFO:teuthology.orchestra.run.smithi085.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T15:13:57.389 INFO:teuthology.orchestra.run.smithi085.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T15:13:57.389 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.389 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default size = 2 2022-11-27T15:13:57.390 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.390 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow primary affinity = true 2022-11-27T15:13:57.390 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow pg remap = true 2022-11-27T15:13:57.390 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on legacy crush tunables = false 2022-11-27T15:13:57.391 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on crush straw calc version zero = false 2022-11-27T15:13:57.391 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on no sortbitwise = false 2022-11-27T15:13:57.391 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on osd down out interval zero = false 2022-11-27T15:13:57.391 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on too few osds = false 2022-11-27T15:13:57.392 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T15:13:57.392 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T15:13:57.392 INFO:teuthology.orchestra.run.smithi085.stdout: mon_allow_pool_size_one = true 2022-11-27T15:13:57.392 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.393 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T15:13:57.393 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.393 INFO:teuthology.orchestra.run.smithi085.stdout: osd default data pool replay window = 5 2022-11-27T15:13:57.393 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.394 INFO:teuthology.orchestra.run.smithi085.stdout: mon allow pool delete = true 2022-11-27T15:13:57.394 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.394 INFO:teuthology.orchestra.run.smithi085.stdout: mon cluster log file level = debug 2022-11-27T15:13:57.394 INFO:teuthology.orchestra.run.smithi085.stdout: debug asserts on shutdown = true 2022-11-27T15:13:57.395 INFO:teuthology.orchestra.run.smithi085.stdout: mon health detail to clog = false 2022-11-27T15:13:57.395 INFO:teuthology.orchestra.run.smithi085.stdout: mon host = "172.21.15.35,172.21.15.85,[v2:172.21.15.85:3301,v1:172.21.15.85:6790]" 2022-11-27T15:13:57.395 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on skipped message = False 2022-11-27T15:13:57.395 INFO:teuthology.orchestra.run.smithi085.stdout: fsid = 1a1c2d0e-bda9-4965-9029-a4d82eabd38a 2022-11-27T15:13:57.396 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.396 INFO:teuthology.orchestra.run.smithi085.stdout:[osd] 2022-11-27T15:13:57.396 INFO:teuthology.orchestra.run.smithi085.stdout: osd journal size = 100 2022-11-27T15:13:57.396 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.397 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub load threshold = 5.0 2022-11-27T15:13:57.397 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub max interval = 600 2022-11-27T15:13:57.397 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.397 INFO:teuthology.orchestra.run.smithi085.stdout: osd recover clone overlap = true 2022-11-27T15:13:57.398 INFO:teuthology.orchestra.run.smithi085.stdout: osd recovery max chunk = 1048576 2022-11-27T15:13:57.398 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.398 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug shutdown = true 2022-11-27T15:13:57.399 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug op order = true 2022-11-27T15:13:57.399 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug verify stray on activate = true 2022-11-27T15:13:57.399 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.399 INFO:teuthology.orchestra.run.smithi085.stdout: osd open classes on start = true 2022-11-27T15:13:57.400 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug pg log writeout = true 2022-11-27T15:13:57.400 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.400 INFO:teuthology.orchestra.run.smithi085.stdout: osd deep scrub update digest min age = 30 2022-11-27T15:13:57.401 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.401 INFO:teuthology.orchestra.run.smithi085.stdout: osd map max advance = 10 2022-11-27T15:13:57.401 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.401 INFO:teuthology.orchestra.run.smithi085.stdout: journal zero on create = true 2022-11-27T15:13:57.402 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.402 INFO:teuthology.orchestra.run.smithi085.stdout: filestore ondisk finisher threads = 3 2022-11-27T15:13:57.402 INFO:teuthology.orchestra.run.smithi085.stdout: filestore apply finisher threads = 3 2022-11-27T15:13:57.403 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.403 INFO:teuthology.orchestra.run.smithi085.stdout: bdev debug aio = true 2022-11-27T15:13:57.403 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug misdirected ops = true 2022-11-27T15:13:57.404 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore block size = 96636764160 2022-11-27T15:13:57.404 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore fsck on mount = True 2022-11-27T15:13:57.404 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluefs = 20 2022-11-27T15:13:57.405 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluestore = 20 2022-11-27T15:13:57.405 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-11-27T15:13:57.405 INFO:teuthology.orchestra.run.smithi085.stdout: debug osd = 20 2022-11-27T15:13:57.405 INFO:teuthology.orchestra.run.smithi085.stdout: debug rocksdb = 10 2022-11-27T15:13:57.406 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T15:13:57.406 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd full ratio = 0.9 2022-11-27T15:13:57.406 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd nearfull ratio = 0.8 2022-11-27T15:13:57.406 INFO:teuthology.orchestra.run.smithi085.stdout: osd failsafe full ratio = 0.95 2022-11-27T15:13:57.406 INFO:teuthology.orchestra.run.smithi085.stdout: osd objectstore = bluestore 2022-11-27T15:13:57.407 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2022-11-27T15:13:57.407 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.407 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr] 2022-11-27T15:13:57.407 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-11-27T15:13:57.408 INFO:teuthology.orchestra.run.smithi085.stdout: debug mgr = 20 2022-11-27T15:13:57.408 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2022-11-27T15:13:57.408 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2022-11-27T15:13:57.408 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2022-11-27T15:13:57.409 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2022-11-27T15:13:57.409 INFO:teuthology.orchestra.run.smithi085.stdout: mgr/telemetry/nag = false 2022-11-27T15:13:57.409 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.409 INFO:teuthology.orchestra.run.smithi085.stdout:[mon] 2022-11-27T15:13:57.410 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-11-27T15:13:57.410 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2022-11-27T15:13:57.410 INFO:teuthology.orchestra.run.smithi085.stdout: debug paxos = 20 2022-11-27T15:13:57.410 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2022-11-27T15:13:57.411 INFO:teuthology.orchestra.run.smithi085.stdout: mon data avail warn = 5 2022-11-27T15:13:57.411 INFO:teuthology.orchestra.run.smithi085.stdout: mon mgr mkfs grace = 240 2022-11-27T15:13:57.411 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2022-11-27T15:13:57.412 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd reporter subtree level = osd 2022-11-27T15:13:57.412 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd prime pg temp = true 2022-11-27T15:13:57.412 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2022-11-27T15:13:57.412 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.413 INFO:teuthology.orchestra.run.smithi085.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T15:13:57.413 INFO:teuthology.orchestra.run.smithi085.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T15:13:57.413 INFO:teuthology.orchestra.run.smithi085.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T15:13:57.414 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.414 INFO:teuthology.orchestra.run.smithi085.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T15:13:57.414 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T15:13:57.415 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T15:13:57.415 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.415 INFO:teuthology.orchestra.run.smithi085.stdout: # 1m isn't quite enough 2022-11-27T15:13:57.415 INFO:teuthology.orchestra.run.smithi085.stdout: mon_down_mkfs_grace = 2m 2022-11-27T15:13:57.416 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.416 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_filestore_osds = false 2022-11-27T15:13:57.416 INFO:teuthology.orchestra.run.smithi085.stdout: mon op complaint time = 120 2022-11-27T15:13:57.417 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-11-27T15:13:57.417 INFO:teuthology.orchestra.run.smithi085.stdout:[client] 2022-11-27T15:13:57.417 INFO:teuthology.orchestra.run.smithi085.stdout: rgw cache enabled = true 2022-11-27T15:13:57.418 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable ops log = true 2022-11-27T15:13:57.418 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable usage log = true 2022-11-27T15:13:57.418 INFO:teuthology.orchestra.run.smithi085.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T15:13:57.418 INFO:teuthology.orchestra.run.smithi085.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T15:13:57.419 INFO:teuthology.orchestra.run.smithi085.stdout: client mount timeout = 600 2022-11-27T15:13:57.419 INFO:teuthology.orchestra.run.smithi085.stdout: debug client = 20 2022-11-27T15:13:57.419 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-11-27T15:13:57.419 INFO:teuthology.orchestra.run.smithi085.stdout: pid file = "" 2022-11-27T15:13:57.420 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 900 2022-11-27T15:13:57.420 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 900 2022-11-27T15:13:57.420 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.a] 2022-11-27T15:13:57.420 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.b] 2022-11-27T15:13:57.421 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.c] 2022-11-27T15:13:57.421 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.a] 2022-11-27T15:13:57.421 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.c] 2022-11-27T15:13:57.422 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.b] 2022-11-27T15:13:57.422 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.d] 2022-11-27T15:13:57.422 INFO:teuthology.orchestra.run.smithi085.stdout:[mds] 2022-11-27T15:13:57.422 INFO:teuthology.orchestra.run.smithi085.stdout: debug mds = 20 2022-11-27T15:13:57.423 INFO:teuthology.orchestra.run.smithi085.stdout: debug mds balancer = 20 2022-11-27T15:13:57.423 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-11-27T15:13:57.423 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug frag = True 2022-11-27T15:13:57.423 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug scatterstat = True 2022-11-27T15:13:57.424 INFO:teuthology.orchestra.run.smithi085.stdout: mds op complaint time = 180 2022-11-27T15:13:57.424 INFO:teuthology.orchestra.run.smithi085.stdout: mds verify scatter = True 2022-11-27T15:13:57.424 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2022-11-27T15:13:57.424 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 900 2022-11-27T15:13:57.425 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 900 2022-11-27T15:13:57.425 INFO:tasks.ceph:Creating admin key on mon.a... 2022-11-27T15:13:57.426 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-11-27T15:13:57.486 INFO:tasks.ceph:Copying monmap to all nodes... 2022-11-27T15:13:57.486 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:13:57.487 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-11-27T15:13:57.505 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:13:57.505 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-11-27T15:13:57.561 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi035.front.sepia.ceph.com 2022-11-27T15:13:57.562 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:13:57.562 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T15:13:57.562 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T15:13:57.641 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:13:57.642 DEBUG:teuthology.orchestra.run.smithi035:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:13:57.699 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi085.front.sepia.ceph.com 2022-11-27T15:13:57.699 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:13:57.700 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T15:13:57.700 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T15:13:57.743 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:13:57.743 DEBUG:teuthology.orchestra.run.smithi085:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:13:57.803 INFO:tasks.ceph:Setting up mon nodes... 2022-11-27T15:13:57.803 INFO:tasks.ceph:Setting up mgr nodes... 2022-11-27T15:13:57.804 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T15:13:57.864 INFO:teuthology.orchestra.run.smithi035.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T15:13:57.864 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T15:13:57.978 INFO:teuthology.orchestra.run.smithi085.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T15:13:57.982 INFO:tasks.ceph:Setting up mds nodes... 2022-11-27T15:13:57.982 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T15:13:58.039 INFO:teuthology.orchestra.run.smithi035.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T15:13:58.042 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-11-27T15:13:58.070 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T15:13:58.166 INFO:teuthology.orchestra.run.smithi035.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T15:13:58.168 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-11-27T15:13:58.196 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T15:13:58.259 INFO:teuthology.orchestra.run.smithi085.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T15:13:58.262 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-11-27T15:13:58.292 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T15:13:58.388 INFO:teuthology.orchestra.run.smithi085.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T15:13:58.390 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2022-11-27T15:13:58.417 INFO:tasks.ceph_client:Setting up client nodes... 2022-11-27T15:13:58.418 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-11-27T15:13:58.463 INFO:teuthology.orchestra.run.smithi035.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-11-27T15:13:58.477 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2022-11-27T15:13:58.522 INFO:teuthology.orchestra.run.smithi035.stdout:creating /etc/ceph/ceph.client.2.keyring 2022-11-27T15:13:58.535 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-11-27T15:13:58.578 INFO:teuthology.orchestra.run.smithi085.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-11-27T15:13:58.591 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2022-11-27T15:13:58.674 INFO:teuthology.orchestra.run.smithi085.stdout:creating /etc/ceph/ceph.client.3.keyring 2022-11-27T15:13:58.686 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-11-27T15:13:58.687 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi035.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi085.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T15:13:58.688 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-11-27T15:13:58.716 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T15:13:58.717 INFO:tasks.ceph:role: osd.0 2022-11-27T15:13:58.717 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi035.front.sepia.ceph.com 2022-11-27T15:13:58.717 DEBUG:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T15:13:58.791 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:13:58.792 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:13:58.792 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:13:58.792 INFO:teuthology.orchestra.run.smithi035.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:13:58.793 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:13:58.793 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:13:58.794 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:13:58.794 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:13:58.794 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:13:58.795 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:13:59.227 INFO:teuthology.orchestra.run.smithi035.stdout:Discarding blocks...Done. 2022-11-27T15:13:59.229 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2022-11-27T15:13:59.230 DEBUG:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-11-27T15:13:59.350 DEBUG:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-11-27T15:13:59.385 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-11-27T15:13:59.453 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T15:13:59.453 INFO:tasks.ceph:role: osd.1 2022-11-27T15:13:59.454 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi035.front.sepia.ceph.com 2022-11-27T15:13:59.455 DEBUG:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T15:13:59.529 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:13:59.529 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:13:59.529 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:13:59.529 INFO:teuthology.orchestra.run.smithi035.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:13:59.530 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:13:59.530 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:13:59.530 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:13:59.530 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:13:59.531 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:13:59.531 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:13:59.971 INFO:teuthology.orchestra.run.smithi035.stdout:Discarding blocks...Done. 2022-11-27T15:13:59.973 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2022-11-27T15:13:59.974 DEBUG:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-11-27T15:14:00.072 DEBUG:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-11-27T15:14:00.108 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-11-27T15:14:00.177 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T15:14:00.178 INFO:tasks.ceph:role: osd.2 2022-11-27T15:14:00.179 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi035.front.sepia.ceph.com 2022-11-27T15:14:00.180 DEBUG:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T15:14:00.254 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:14:00.255 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:14:00.255 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:14:00.256 INFO:teuthology.orchestra.run.smithi035.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:14:00.256 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:14:00.257 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:14:00.257 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:14:00.257 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:14:00.258 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:14:00.258 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:14:00.695 INFO:teuthology.orchestra.run.smithi035.stdout:Discarding blocks...Done. 2022-11-27T15:14:00.697 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2022-11-27T15:14:00.698 DEBUG:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-11-27T15:14:00.825 DEBUG:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-11-27T15:14:00.862 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-11-27T15:14:00.931 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T15:14:00.932 INFO:tasks.ceph:role: osd.3 2022-11-27T15:14:00.932 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi035.front.sepia.ceph.com 2022-11-27T15:14:00.933 DEBUG:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T15:14:01.009 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:14:01.010 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:14:01.010 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:14:01.011 INFO:teuthology.orchestra.run.smithi035.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:14:01.011 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:14:01.012 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:14:01.012 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:14:01.013 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:14:01.013 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:14:01.013 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:14:01.456 INFO:teuthology.orchestra.run.smithi035.stdout:Discarding blocks...Done. 2022-11-27T15:14:01.459 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2022-11-27T15:14:01.460 DEBUG:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-11-27T15:14:01.576 DEBUG:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-11-27T15:14:01.611 DEBUG:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:14:01.704 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:01.702+0000 7fddb73ff5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-11-27T15:14:01.705 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:01.703+0000 7fddb73ff5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-11-27T15:14:01.705 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:01.703+0000 7fddb73ff5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T15:14:01.706 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:01.703+0000 7fddb73ff5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T15:14:01.706 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:01.703+0000 7fddb73ff5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T15:14:01.707 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:01.705+0000 7fddb73ff5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-11-27T15:14:04.583 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-11-27T15:14:04.617 DEBUG:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:14:04.710 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:04.708+0000 7f3e9dac55c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-11-27T15:14:04.711 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:04.708+0000 7f3e9dac55c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-11-27T15:14:04.711 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:04.708+0000 7f3e9dac55c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T15:14:04.711 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:04.708+0000 7f3e9dac55c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T15:14:04.711 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:04.708+0000 7f3e9dac55c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T15:14:04.712 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:04.709+0000 7f3e9dac55c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-11-27T15:14:07.580 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-11-27T15:14:07.612 DEBUG:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:14:07.706 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:07.704+0000 7f69cca245c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-11-27T15:14:07.707 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:07.704+0000 7f69cca245c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-11-27T15:14:07.707 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:07.704+0000 7f69cca245c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T15:14:07.707 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:07.704+0000 7f69cca245c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T15:14:07.708 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:07.704+0000 7f69cca245c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T15:14:07.708 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:07.705+0000 7f69cca245c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-11-27T15:14:10.565 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-11-27T15:14:10.600 DEBUG:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:14:10.695 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:10.692+0000 7f43d9ee45c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-11-27T15:14:10.695 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:10.692+0000 7f43d9ee45c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-11-27T15:14:10.696 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:10.692+0000 7f43d9ee45c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T15:14:10.696 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:10.692+0000 7f43d9ee45c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T15:14:10.696 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:10.692+0000 7f43d9ee45c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T15:14:10.696 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:10.693+0000 7f43d9ee45c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-11-27T15:14:13.559 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-11-27T15:14:13.593 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-11-27T15:14:13.628 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T15:14:13.628 INFO:tasks.ceph:role: osd.4 2022-11-27T15:14:13.629 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi085.front.sepia.ceph.com 2022-11-27T15:14:13.629 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T15:14:13.706 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:14:13.707 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:14:13.707 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:14:13.708 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:14:13.708 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:14:13.708 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:14:13.709 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:14:13.709 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:14:13.710 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:14:13.710 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:14:14.136 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2022-11-27T15:14:14.138 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2022-11-27T15:14:14.139 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-11-27T15:14:14.315 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-11-27T15:14:14.353 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-11-27T15:14:14.422 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T15:14:14.423 INFO:tasks.ceph:role: osd.5 2022-11-27T15:14:14.423 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi085.front.sepia.ceph.com 2022-11-27T15:14:14.424 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T15:14:14.500 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:14:14.500 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:14:14.500 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:14:14.500 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:14:14.501 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:14:14.501 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:14:14.501 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:14:14.501 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:14:14.502 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:14:14.502 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:14:14.920 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2022-11-27T15:14:14.922 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2022-11-27T15:14:14.923 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-11-27T15:14:15.038 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-11-27T15:14:15.072 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-11-27T15:14:15.141 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T15:14:15.141 INFO:tasks.ceph:role: osd.6 2022-11-27T15:14:15.142 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi085.front.sepia.ceph.com 2022-11-27T15:14:15.142 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T15:14:15.217 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:14:15.218 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:14:15.218 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:14:15.218 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:14:15.218 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:14:15.219 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:14:15.219 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:14:15.219 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:14:15.219 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:14:15.220 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:14:15.661 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2022-11-27T15:14:15.663 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2022-11-27T15:14:15.663 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-11-27T15:14:15.824 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-11-27T15:14:15.860 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-11-27T15:14:15.930 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T15:14:15.931 INFO:tasks.ceph:role: osd.7 2022-11-27T15:14:15.931 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi085.front.sepia.ceph.com 2022-11-27T15:14:15.932 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T15:14:16.011 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:14:16.011 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:14:16.012 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:14:16.012 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:14:16.012 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:14:16.012 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:14:16.013 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:14:16.013 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:14:16.013 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:14:16.013 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:14:16.449 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2022-11-27T15:14:16.451 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2022-11-27T15:14:16.451 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-11-27T15:14:16.639 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-11-27T15:14:16.678 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:14:16.773 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:16.771+0000 7f7fae4bb5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-11-27T15:14:16.773 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:16.771+0000 7f7fae4bb5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-11-27T15:14:16.774 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:16.771+0000 7f7fae4bb5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T15:14:16.774 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:16.771+0000 7f7fae4bb5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T15:14:16.774 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:16.771+0000 7f7fae4bb5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T15:14:16.775 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:16.772+0000 7f7fae4bb5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-11-27T15:14:19.646 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-11-27T15:14:19.680 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:14:19.772 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:19.770+0000 7f6b135715c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-11-27T15:14:19.773 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:19.771+0000 7f6b135715c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-11-27T15:14:19.773 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:19.771+0000 7f6b135715c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T15:14:19.773 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:19.771+0000 7f6b135715c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T15:14:19.773 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:19.771+0000 7f6b135715c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T15:14:19.774 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:19.771+0000 7f6b135715c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-11-27T15:14:22.636 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-11-27T15:14:22.667 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:14:22.757 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:22.755+0000 7ff1a7a175c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-11-27T15:14:22.757 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:22.755+0000 7ff1a7a175c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-11-27T15:14:22.757 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:22.755+0000 7ff1a7a175c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T15:14:22.758 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:22.755+0000 7ff1a7a175c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T15:14:22.758 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:22.755+0000 7ff1a7a175c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T15:14:22.758 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:22.755+0000 7ff1a7a175c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-11-27T15:14:25.209 DEBUG:teuthology.orchestra.run.smithi035:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T15:14:25.214 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T15:14:25.615 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-11-27T15:14:25.647 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:14:25.740 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:25.737+0000 7f4216a535c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-11-27T15:14:25.740 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:25.738+0000 7f4216a535c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-11-27T15:14:25.741 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:25.738+0000 7f4216a535c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T15:14:25.741 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:25.738+0000 7f4216a535c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T15:14:25.741 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:25.738+0000 7f4216a535c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T15:14:25.741 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:25.738+0000 7f4216a535c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-11-27T15:14:28.603 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-11-27T15:14:28.638 INFO:tasks.ceph:Reading keys from all nodes... 2022-11-27T15:14:28.639 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:28.640 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-11-27T15:14:28.676 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:28.676 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-11-27T15:14:28.746 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:28.747 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-11-27T15:14:28.816 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:28.817 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-11-27T15:14:28.886 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:28.886 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-11-27T15:14:28.957 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:28.958 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-11-27T15:14:29.027 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:29.028 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-11-27T15:14:29.097 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:29.097 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-11-27T15:14:29.129 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:29.129 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-11-27T15:14:29.201 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:29.201 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2022-11-27T15:14:29.271 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:29.271 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2022-11-27T15:14:29.339 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:29.339 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2022-11-27T15:14:29.407 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:29.408 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2022-11-27T15:14:29.477 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:29.478 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2022-11-27T15:14:29.543 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:29.543 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-11-27T15:14:29.565 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:29.566 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2022-11-27T15:14:29.627 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:29.627 DEBUG:teuthology.orchestra.run.smithi085:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-11-27T15:14:29.649 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:29.650 DEBUG:teuthology.orchestra.run.smithi085:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2022-11-27T15:14:29.706 INFO:tasks.ceph:Adding keys to all mons... 2022-11-27T15:14:29.707 DEBUG:teuthology.orchestra.run.smithi035:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T15:14:29.709 DEBUG:teuthology.orchestra.run.smithi085:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T15:14:29.750 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr.y] 2022-11-27T15:14:29.751 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC1foNjthY0MxAAvjaZMko0YLRleok6iMYu2Q== 2022-11-27T15:14:29.751 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.a] 2022-11-27T15:14:29.752 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC2foNjPUhaAhAAk17qGMM5IjepHrFpUB41/A== 2022-11-27T15:14:29.752 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.c] 2022-11-27T15:14:29.752 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC2foNjugnjCRAAaba3OQJVsOzvaGtjw/DHhQ== 2022-11-27T15:14:29.753 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.0] 2022-11-27T15:14:29.753 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC5foNj17z5KRAAg9bxKdAbPlDaGE6WOKutNg== 2022-11-27T15:14:29.753 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.1] 2022-11-27T15:14:29.753 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC8foNjUL1OKhAAUJOH0psu6jGlv4faLHnWdg== 2022-11-27T15:14:29.754 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.2] 2022-11-27T15:14:29.754 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC/foNjTeQSKhAAmmPcLanL0qIoR8+VNT7/iw== 2022-11-27T15:14:29.754 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.3] 2022-11-27T15:14:29.754 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDCfoNjbJBeKRAADVZIGSsjmInspBxGRe7AIw== 2022-11-27T15:14:29.755 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr.x] 2022-11-27T15:14:29.755 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC1foNjMfRUOhAAy9YWFMcHdWwe6Pibj53p3g== 2022-11-27T15:14:29.755 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.b] 2022-11-27T15:14:29.755 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC2foNjwhNsDxAAe2WTzo/isIzOYgg2EQk/eg== 2022-11-27T15:14:29.756 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.d] 2022-11-27T15:14:29.756 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC2foNj8REgFxAAbb6ivgN0OJfcio3BDSO6Dg== 2022-11-27T15:14:29.756 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.4] 2022-11-27T15:14:29.759 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDIfoNj618NLhAAJWrrkjLZ0KU8b/Rc6KNjjA== 2022-11-27T15:14:29.759 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.5] 2022-11-27T15:14:29.760 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDLfoNjsooCLhAA/qOomOV33UmVJEZzC2rqdw== 2022-11-27T15:14:29.760 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.6] 2022-11-27T15:14:29.760 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDOfoNjHw8SLRAAXTnv82YJiho0jgDPxK+AoQ== 2022-11-27T15:14:29.760 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.7] 2022-11-27T15:14:29.761 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQDRfoNjcRcLLBAAt20jgLfIgEOnJfypSnyyLg== 2022-11-27T15:14:29.761 INFO:teuthology.orchestra.run.smithi035.stdout:[client.0] 2022-11-27T15:14:29.761 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC2foNjyJWhGxAAhK87CKg5J59hrLftxL+u3A== 2022-11-27T15:14:29.761 INFO:teuthology.orchestra.run.smithi035.stdout:[client.2] 2022-11-27T15:14:29.762 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC2foNjqnYbHxAA67DH0F37D3g+mNW3xZXeOA== 2022-11-27T15:14:29.762 INFO:teuthology.orchestra.run.smithi035.stdout:[client.1] 2022-11-27T15:14:29.762 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC2foNjgOl7IhAAn7mfk1C8TCj0SPTng71kcg== 2022-11-27T15:14:29.762 INFO:teuthology.orchestra.run.smithi035.stdout:[client.3] 2022-11-27T15:14:29.762 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQC2foNjxCoqKBAAWZ7+dHV7WRLVBoypQ2dN3Q== 2022-11-27T15:14:29.771 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr.y] 2022-11-27T15:14:29.771 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC1foNjthY0MxAAvjaZMko0YLRleok6iMYu2Q== 2022-11-27T15:14:29.771 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.a] 2022-11-27T15:14:29.771 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC2foNjPUhaAhAAk17qGMM5IjepHrFpUB41/A== 2022-11-27T15:14:29.772 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.c] 2022-11-27T15:14:29.772 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC2foNjugnjCRAAaba3OQJVsOzvaGtjw/DHhQ== 2022-11-27T15:14:29.772 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.0] 2022-11-27T15:14:29.772 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC5foNj17z5KRAAg9bxKdAbPlDaGE6WOKutNg== 2022-11-27T15:14:29.773 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.1] 2022-11-27T15:14:29.773 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC8foNjUL1OKhAAUJOH0psu6jGlv4faLHnWdg== 2022-11-27T15:14:29.773 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.2] 2022-11-27T15:14:29.773 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC/foNjTeQSKhAAmmPcLanL0qIoR8+VNT7/iw== 2022-11-27T15:14:29.774 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.3] 2022-11-27T15:14:29.774 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQDCfoNjbJBeKRAADVZIGSsjmInspBxGRe7AIw== 2022-11-27T15:14:29.774 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr.x] 2022-11-27T15:14:29.774 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC1foNjMfRUOhAAy9YWFMcHdWwe6Pibj53p3g== 2022-11-27T15:14:29.774 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.b] 2022-11-27T15:14:29.775 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC2foNjwhNsDxAAe2WTzo/isIzOYgg2EQk/eg== 2022-11-27T15:14:29.775 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.d] 2022-11-27T15:14:29.775 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC2foNj8REgFxAAbb6ivgN0OJfcio3BDSO6Dg== 2022-11-27T15:14:29.775 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.4] 2022-11-27T15:14:29.776 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQDIfoNj618NLhAAJWrrkjLZ0KU8b/Rc6KNjjA== 2022-11-27T15:14:29.776 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.5] 2022-11-27T15:14:29.777 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQDLfoNjsooCLhAA/qOomOV33UmVJEZzC2rqdw== 2022-11-27T15:14:29.777 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.6] 2022-11-27T15:14:29.777 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQDOfoNjHw8SLRAAXTnv82YJiho0jgDPxK+AoQ== 2022-11-27T15:14:29.777 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.7] 2022-11-27T15:14:29.778 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQDRfoNjcRcLLBAAt20jgLfIgEOnJfypSnyyLg== 2022-11-27T15:14:29.778 INFO:teuthology.orchestra.run.smithi085.stdout:[client.0] 2022-11-27T15:14:29.778 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC2foNjyJWhGxAAhK87CKg5J59hrLftxL+u3A== 2022-11-27T15:14:29.778 INFO:teuthology.orchestra.run.smithi085.stdout:[client.2] 2022-11-27T15:14:29.779 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC2foNjqnYbHxAA67DH0F37D3g+mNW3xZXeOA== 2022-11-27T15:14:29.779 INFO:teuthology.orchestra.run.smithi085.stdout:[client.1] 2022-11-27T15:14:29.779 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC2foNjgOl7IhAAn7mfk1C8TCj0SPTng71kcg== 2022-11-27T15:14:29.779 INFO:teuthology.orchestra.run.smithi085.stdout:[client.3] 2022-11-27T15:14:29.780 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC2foNjxCoqKBAAWZ7+dHV7WRLVBoypQ2dN3Q== 2022-11-27T15:14:29.781 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T15:14:29.807 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T15:14:29.873 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:14:29.877 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:14:29.936 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:14:29.940 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:14:29.998 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.001 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.053 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.057 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.110 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.113 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.168 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.173 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.226 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T15:14:30.230 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T15:14:30.283 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:14:30.286 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:14:30.343 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:14:30.346 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:14:30.403 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.406 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.460 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.463 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.516 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.518 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.571 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.574 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:14:30.628 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T15:14:30.631 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T15:14:30.687 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T15:14:30.690 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T15:14:30.748 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T15:14:30.750 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T15:14:30.808 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T15:14:30.811 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T15:14:30.869 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-11-27T15:14:30.869 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-11-27T15:14:30.897 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T15:14:30.996 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-11-27T15:14:31.024 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-11-27T15:14:31.052 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T15:14:31.150 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-11-27T15:14:31.177 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-11-27T15:14:31.244 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T15:14:31.341 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-11-27T15:14:31.369 DEBUG:teuthology.orchestra.run.smithi035:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:14:31.373 DEBUG:teuthology.orchestra.run.smithi085:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:14:31.426 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-11-27T15:14:31.427 INFO:tasks.ceph.mon.a:Restarting daemon 2022-11-27T15:14:31.428 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-11-27T15:14:31.431 INFO:tasks.ceph.mon.a:Started 2022-11-27T15:14:31.431 INFO:tasks.ceph.mon.b:Restarting daemon 2022-11-27T15:14:31.432 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-11-27T15:14:31.470 INFO:tasks.ceph.mon.b:Started 2022-11-27T15:14:31.470 INFO:tasks.ceph.mon.c:Restarting daemon 2022-11-27T15:14:31.471 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-11-27T15:14:31.474 INFO:tasks.ceph.mon.c:Started 2022-11-27T15:14:31.475 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-11-27T15:14:31.476 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-11-27T15:14:31.477 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-11-27T15:14:31.480 INFO:tasks.ceph.mgr.y:Started 2022-11-27T15:14:31.481 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-11-27T15:14:31.481 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-11-27T15:14:31.485 INFO:tasks.ceph.mgr.x:Started 2022-11-27T15:14:31.486 INFO:tasks.ceph:Setting crush tunables to default 2022-11-27T15:14:31.487 DEBUG:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd crush tunables default 2022-11-27T15:14:31.615 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.614+0000 7f6850abd700 1 Processor -- start 2022-11-27T15:14:31.616 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.614+0000 7f6850abd700 1 -- start start 2022-11-27T15:14:31.618 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.615+0000 7f6850abd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 0x7f684c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:31.618 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.615+0000 7f6850abd700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f684c15cc20 0x7f684c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:31.618 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.615+0000 7f6850abd700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0ce5f0 con 0x7f684c158630 2022-11-27T15:14:31.619 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.615+0000 7f6850abd700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684c0ce0f0 con 0x7f684c15cc20 2022-11-27T15:14:31.619 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.615+0000 7f6850abd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0cc810 con 0x7f684c0cd540 2022-11-27T15:14:31.619 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.615+0000 7f684a59c700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f684c158630 0x7f684c0cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50750/0 (socket says 172.21.15.35:50750) 2022-11-27T15:14:31.619 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.615+0000 7f684a59c700 1 -- 172.21.15.35:0/2359470833 learned_addr learned my addr 172.21.15.35:0/2359470833 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-27T15:14:31.620 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.615+0000 7f684ad9d700 1 --2- 172.21.15.35:0/2359470833 >> v2:172.21.15.85:3300/0 conn(0x7f684c15cc20 0x7f684c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:31.620 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.615+0000 7f6849d9b700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 0x7f684c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:31.620 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.616+0000 7f6849d9b700 1 -- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 msgr2=0x7f684c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T15:14:31.620 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.616+0000 7f6849d9b700 1 -- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 msgr2=0x7f684c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T15:14:31.621 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.616+0000 7f6849d9b700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 0x7f684c0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T15:14:31.621 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.616+0000 7f684ad9d700 1 -- 172.21.15.35:0/2359470833 >> v2:172.21.15.85:3300/0 conn(0x7f684c15cc20 msgr2=0x7f684c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-11-27T15:14:31.621 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.616+0000 7f684ad9d700 1 -- 172.21.15.35:0/2359470833 >> v2:172.21.15.85:3300/0 conn(0x7f684c15cc20 msgr2=0x7f684c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T15:14:31.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.616+0000 7f684ad9d700 1 --2- 172.21.15.35:0/2359470833 >> v2:172.21.15.85:3300/0 conn(0x7f684c15cc20 0x7f684c0b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T15:14:31.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.616+0000 7f6849d9b700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 0x7f684c0b06c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T15:14:31.623 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.616+0000 7f684ad9d700 1 --2- 172.21.15.35:0/2359470833 >> v2:172.21.15.85:3300/0 conn(0x7f684c15cc20 0x7f684c0b0c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T15:14:31.812 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.811+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806381405 0 0) 0x7f684c0ce5f0 con 0x7f684c158630 2022-11-27T15:14:31.812 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.811+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68300035b0 con 0x7f684c158630 2022-11-27T15:14:31.816 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 <== mon.0 v1:172.21.15.35:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3528028072 0 0) 0x7f68300035b0 con 0x7f684c158630 2022-11-27T15:14:31.817 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 msgr2=0x7f684c0b06c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T15:14:31.817 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 0x7f684c0b06c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:31.817 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 >> v2:172.21.15.85:3300/0 conn(0x7f684c15cc20 msgr2=0x7f684c0b0c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T15:14:31.817 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 --2- 172.21.15.35:0/2359470833 >> v2:172.21.15.85:3300/0 conn(0x7f684c15cc20 0x7f684c0b0c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:31.818 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 --> v1:172.21.15.35:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0ce310 con 0x7f684c158630 2022-11-27T15:14:31.818 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 <== mon.0 v1:172.21.15.35:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1516710402 0 0) 0x7f6844001b70 con 0x7f684c158630 2022-11-27T15:14:31.818 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 >> v1:172.21.15.35:6789/0 conn(0x7f684c158630 legacy=0x7f684c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:31.818 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 0x7f6830003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:31.819 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6830004030 0x7f6830004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:31.819 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004d90 0x7f6830005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:31.819 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0ce0f0 con 0x7f6830004d90 2022-11-27T15:14:31.819 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0cc810 con 0x7f6830004030 2022-11-27T15:14:31.819 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683000bb10 con 0x7f684c0cd540 2022-11-27T15:14:31.820 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f6849d9b700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6830004030 0x7f6830004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:31.820 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684ad9d700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004d90 0x7f6830005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:31.820 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.813+0000 7f684a59c700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 0x7f6830003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:31.820 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.814+0000 7f6849d9b700 1 -- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 msgr2=0x7f6830003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:31.821 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.814+0000 7f6849d9b700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c0cd540 0x7f6830003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:31.821 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.814+0000 7f6849d9b700 1 -- 172.21.15.35:0/2359470833 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004d90 msgr2=0x7f6830005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:31.821 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.814+0000 7f6849d9b700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004d90 0x7f6830005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:31.821 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.814+0000 7f6849d9b700 1 -- 172.21.15.35:0/2359470833 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68300035b0 con 0x7f6830004030 2022-11-27T15:14:31.821 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.814+0000 7f684ad9d700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004d90 0x7f6830005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:31.822 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.814+0000 7f6849d9b700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6830004030 0x7f6830004850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f684c0cd930 tx=0x7f6830005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=461f069125556770 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:31.822 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.814+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684000c070 con 0x7f6830004030 2022-11-27T15:14:31.822 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.814+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840004020 con 0x7f6830004030 2022-11-27T15:14:31.822 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.814+0000 7f684959a700 1 -- 172.21.15.35:0/2359470833 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400043d0 con 0x7f6830004030 2022-11-27T15:14:31.823 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.815+0000 7f6850abd700 1 -- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6830004030 msgr2=0x7f6830004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:31.823 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.815+0000 7f6850abd700 1 --2- 172.21.15.35:0/2359470833 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6830004030 0x7f6830004850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f684c0cd930 tx=0x7f6830005880 comp rx=0 tx=0).stop 2022-11-27T15:14:31.823 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.815+0000 7f684ad9d700 1 -- 172.21.15.35:0/2359470833 reap_dead start 2022-11-27T15:14:31.823 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.815+0000 7f6850abd700 1 -- 172.21.15.35:0/2359470833 shutdown_connections 2022-11-27T15:14:31.824 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.815+0000 7f6850abd700 1 -- 172.21.15.35:0/2359470833 >> 172.21.15.35:0/2359470833 conn(0x7f684c06d9c0 msgr2=0x7f684c0cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:31.827 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.815+0000 7f6850abd700 1 -- 172.21.15.35:0/2359470833 shutdown_connections 2022-11-27T15:14:31.827 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.815+0000 7f6850abd700 1 -- 172.21.15.35:0/2359470833 wait complete. 2022-11-27T15:14:31.827 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.817+0000 7f6850abd700 1 Processor -- start 2022-11-27T15:14:31.828 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.817+0000 7f6850abd700 1 -- start start 2022-11-27T15:14:31.828 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f6850abd700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004030 0x7f684c149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:31.828 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f6850abd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f684c0b1a70 0x7f684c0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:31.828 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f6850abd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c1565c0 0x7f684c156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:31.829 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f6850abd700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0b25e0 con 0x7f6830004030 2022-11-27T15:14:31.829 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f6850abd700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1efab0 con 0x7f684c0b1a70 2022-11-27T15:14:31.829 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f6850abd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c156f70 con 0x7f684c1565c0 2022-11-27T15:14:31.830 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684a59c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004030 0x7f684c149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:31.830 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684a59c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004030 0x7f684c149080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39384/0 (socket says 172.21.15.35:39384) 2022-11-27T15:14:31.830 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684a59c700 1 -- 172.21.15.35:0/3103364064 learned_addr learned my addr 172.21.15.35:0/3103364064 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:31.830 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f6849d9b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f684c0b1a70 0x7f684c0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:31.830 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684ad9d700 1 --2- 172.21.15.35:0/3103364064 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c1565c0 0x7f684c156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:31.831 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684a59c700 1 -- 172.21.15.35:0/3103364064 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c1565c0 msgr2=0x7f684c156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:31.831 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684a59c700 1 --2- 172.21.15.35:0/3103364064 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c1565c0 0x7f684c156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:31.831 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684a59c700 1 -- 172.21.15.35:0/3103364064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f684c0b1a70 msgr2=0x7f684c0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:31.831 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684a59c700 1 --2- 172.21.15.35:0/3103364064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f684c0b1a70 0x7f684c0b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:31.832 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684a59c700 1 -- 172.21.15.35:0/3103364064 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c1495c0 con 0x7f6830004030 2022-11-27T15:14:31.832 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684ad9d700 1 --2- 172.21.15.35:0/3103364064 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c1565c0 0x7f684c156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T15:14:31.832 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.818+0000 7f684a59c700 1 --2- 172.21.15.35:0/3103364064 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004030 0x7f684c149080 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6844001aa0 tx=0x7f684400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:31.832 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.819+0000 7f683affd700 1 -- 172.21.15.35:0/3103364064 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844001bf0 con 0x7f6830004030 2022-11-27T15:14:31.833 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.819+0000 7f683affd700 1 -- 172.21.15.35:0/3103364064 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844005dc0 con 0x7f6830004030 2022-11-27T15:14:31.833 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.819+0000 7f6850abd700 1 -- 172.21.15.35:0/3103364064 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c1ef110 con 0x7f6830004030 2022-11-27T15:14:31.833 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.819+0000 7f6850abd700 1 -- 172.21.15.35:0/3103364064 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c1ef6d0 con 0x7f6830004030 2022-11-27T15:14:31.833 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.819+0000 7f683affd700 1 -- 172.21.15.35:0/3103364064 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400dc80 con 0x7f6830004030 2022-11-27T15:14:31.834 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.820+0000 7f683affd700 1 -- 172.21.15.35:0/3103364064 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f68440073c0 con 0x7f6830004030 2022-11-27T15:14:31.834 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.820+0000 7f683affd700 1 -- 172.21.15.35:0/3103364064 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f684401d730 con 0x7f6830004030 2022-11-27T15:14:31.834 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.820+0000 7f6850abd700 1 -- 172.21.15.35:0/3103364064 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682c0052f0 con 0x7f6830004030 2022-11-27T15:14:31.834 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.822+0000 7f683affd700 1 -- 172.21.15.35:0/3103364064 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f684400dde0 con 0x7f6830004030 2022-11-27T15:14:31.900 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.899+0000 7f6850abd700 1 -- 172.21.15.35:0/3103364064 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f682c005870 con 0x7f6830004030 2022-11-27T15:14:31.907 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.906+0000 7f683affd700 1 -- 172.21.15.35:0/3103364064 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6844007830 con 0x7f6830004030 2022-11-27T15:14:31.908 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.907+0000 7f6850abd700 1 -- 172.21.15.35:0/3103364064 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004030 msgr2=0x7f684c149080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:31.909 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.907+0000 7f6850abd700 1 --2- 172.21.15.35:0/3103364064 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004030 0x7f684c149080 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6844001aa0 tx=0x7f684400d430 comp rx=0 tx=0).stop 2022-11-27T15:14:31.909 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.907+0000 7f6850abd700 1 -- 172.21.15.35:0/3103364064 shutdown_connections 2022-11-27T15:14:31.909 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.907+0000 7f6850abd700 1 --2- 172.21.15.35:0/3103364064 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f684c1565c0 0x7f684c156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:31.910 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.907+0000 7f6850abd700 1 --2- 172.21.15.35:0/3103364064 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6830004030 0x7f684c149080 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:31.910 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.907+0000 7f6850abd700 1 --2- 172.21.15.35:0/3103364064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f684c0b1a70 0x7f684c0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:31.910 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.907+0000 7f6850abd700 1 -- 172.21.15.35:0/3103364064 >> 172.21.15.35:0/3103364064 conn(0x7f684c06d9c0 msgr2=0x7f684c0cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:31.910 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.907+0000 7f6850abd700 1 -- 172.21.15.35:0/3103364064 shutdown_connections 2022-11-27T15:14:31.911 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:31.907+0000 7f6850abd700 1 -- 172.21.15.35:0/3103364064 wait complete. 2022-11-27T15:14:31.911 INFO:teuthology.orchestra.run.smithi035.stderr:adjusted tunables profile to default 2022-11-27T15:14:31.921 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-11-27T15:14:31.922 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:31.922 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-11-27T15:14:31.926 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:31.925+0000 7f6e4d15a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T15:14:31.934 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:31.934+0000 7f2b58316200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T15:14:31.949 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:31.950 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-11-27T15:14:31.989 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:31.988+0000 7f6e4d15a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T15:14:32.002 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:32.002+0000 7f2b58316200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T15:14:32.017 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:32.018 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-11-27T15:14:32.062 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:32.061+0000 7f6e4d15a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T15:14:32.081 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:32.080+0000 7f2b58316200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T15:14:32.086 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-27T15:14:32.086 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-11-27T15:14:32.139 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:32.138+0000 7f6e4d15a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T15:14:32.154 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:32.155 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2022-11-27T15:14:32.165 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:32.165+0000 7f2b58316200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T15:14:32.182 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:32.183 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2022-11-27T15:14:32.250 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:32.251 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2022-11-27T15:14:32.289 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:32.288+0000 7f6e4d15a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T15:14:32.312 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:32.312+0000 7f2b58316200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T15:14:32.315 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-11-27T15:14:32.316 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2022-11-27T15:14:32.381 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 92e0fd37-a176-44da-847e-2bc153faa513 0 2022-11-27T15:14:32.539 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.538+0000 7faf09a75700 1 Processor -- start 2022-11-27T15:14:32.540 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.538+0000 7faf09a75700 1 -- start start 2022-11-27T15:14:32.543 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf09a75700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7faf0413f890 0x7faf0413cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:32.543 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf09a75700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7faf04069410 0x7faf04069890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:32.543 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf09a75700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0415d0d0 con 0x7faf0413f4c0 2022-11-27T15:14:32.543 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf09a75700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf04069dd0 con 0x7faf0413f890 2022-11-27T15:14:32.544 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf09a75700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf0415d3b0 con 0x7faf04069410 2022-11-27T15:14:32.544 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf03fff700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7faf04069410 0x7faf04069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:32.544 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf03fff700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7faf04069410 0x7faf04069890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:39042/0 (socket says 172.21.15.85:39042) 2022-11-27T15:14:32.544 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf03fff700 1 -- 172.21.15.85:0/1873816064 learned_addr learned my addr 172.21.15.85:0/1873816064 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:32.545 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf02ffd700 1 --2- 172.21.15.85:0/1873816064 >> v2:172.21.15.35:3300/0 conn(0x7faf0413f890 0x7faf0413cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:32.545 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf03fff700 1 -- 172.21.15.85:0/1873816064 >> v2:172.21.15.35:3300/0 conn(0x7faf0413f890 msgr2=0x7faf0413cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:32.545 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf03fff700 1 --2- 172.21.15.85:0/1873816064 >> v2:172.21.15.35:3300/0 conn(0x7faf0413f890 0x7faf0413cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.545 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf03fff700 1 -- 172.21.15.85:0/1873816064 >> v1:172.21.15.35:6789/0 conn(0x7faf0413f4c0 legacy=0x7faf0406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.545 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf03fff700 1 -- 172.21.15.85:0/1873816064 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0415d690 con 0x7faf04069410 2022-11-27T15:14:32.546 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.539+0000 7faf03fff700 1 --2- 172.21.15.85:0/1873816064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7faf04069410 0x7faf04069890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf0415df30 tx=0x7faefc009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d8be17efee9891 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:32.546 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf027fc700 1 -- 172.21.15.85:0/1873816064 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00b870 con 0x7faf04069410 2022-11-27T15:14:32.546 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf027fc700 1 -- 172.21.15.85:0/1873816064 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc00b9d0 con 0x7faf04069410 2022-11-27T15:14:32.546 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf027fc700 1 -- 172.21.15.85:0/1873816064 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00bd60 con 0x7faf04069410 2022-11-27T15:14:32.547 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf09a75700 1 -- 172.21.15.85:0/1873816064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7faf04069410 msgr2=0x7faf04069890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.548 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf09a75700 1 --2- 172.21.15.85:0/1873816064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7faf04069410 0x7faf04069890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf0415df30 tx=0x7faefc009bd0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.548 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf09a75700 1 -- 172.21.15.85:0/1873816064 shutdown_connections 2022-11-27T15:14:32.548 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf09a75700 1 --2- 172.21.15.85:0/1873816064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7faf04069410 0x7faf04069890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.549 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf09a75700 1 --2- 172.21.15.85:0/1873816064 >> v2:172.21.15.35:3300/0 conn(0x7faf0413f890 0x7faf0413cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.549 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf09a75700 1 -- 172.21.15.85:0/1873816064 >> 172.21.15.85:0/1873816064 conn(0x7faf040cb2d0 msgr2=0x7faf040cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:32.549 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf09a75700 1 -- 172.21.15.85:0/1873816064 shutdown_connections 2022-11-27T15:14:32.549 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.540+0000 7faf09a75700 1 -- 172.21.15.85:0/1873816064 wait complete. 2022-11-27T15:14:32.550 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.542+0000 7faf09a75700 1 Processor -- start 2022-11-27T15:14:32.556 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.542+0000 7faf09a75700 1 -- start start 2022-11-27T15:14:32.556 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf09a75700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7faf04069410 0x7faf0414f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:32.556 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf09a75700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7faf0413f4c0 0x7faf040c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:32.557 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf09a75700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7faf0413f890 0x7faf040c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:32.557 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf09a75700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0414fd30 con 0x7faf0413f890 2022-11-27T15:14:32.558 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf09a75700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf040cb790 con 0x7faf0413f4c0 2022-11-27T15:14:32.558 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf09a75700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf040cba70 con 0x7faf04069410 2022-11-27T15:14:32.559 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf037fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7faf04069410 0x7faf0414f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:32.559 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf02ffd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7faf0413f4c0 0x7faf040c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:32.560 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf037fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7faf04069410 0x7faf0414f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:36930/0 (socket says 172.21.15.85:36930) 2022-11-27T15:14:32.560 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf037fe700 1 -- 172.21.15.85:0/4101372480 learned_addr learned my addr 172.21.15.85:0/4101372480 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:32.561 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf03fff700 1 --2- 172.21.15.85:0/4101372480 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7faf0413f890 0x7faf040c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:32.561 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf037fe700 1 -- 172.21.15.85:0/4101372480 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7faf0413f4c0 msgr2=0x7faf040c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.562 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf037fe700 1 --2- 172.21.15.85:0/4101372480 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7faf0413f4c0 0x7faf040c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.562 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf037fe700 1 -- 172.21.15.85:0/4101372480 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7faf0413f890 msgr2=0x7faf040c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.562 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf037fe700 1 --2- 172.21.15.85:0/4101372480 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7faf0413f890 0x7faf040c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.563 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf037fe700 1 -- 172.21.15.85:0/4101372480 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc009550 con 0x7faf04069410 2022-11-27T15:14:32.563 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf037fe700 1 --2- 172.21.15.85:0/4101372480 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7faf04069410 0x7faf0414f7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faef4001ed0 tx=0x7faef40021e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:32.564 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faeebfff700 1 -- 172.21.15.85:0/4101372480 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4003c00 con 0x7faf04069410 2022-11-27T15:14:32.564 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faeebfff700 1 -- 172.21.15.85:0/4101372480 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef4003d60 con 0x7faf04069410 2022-11-27T15:14:32.565 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faf09a75700 1 -- 172.21.15.85:0/4101372480 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf040cbd50 con 0x7faf04069410 2022-11-27T15:14:32.565 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.543+0000 7faeebfff700 1 -- 172.21.15.85:0/4101372480 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef400ab40 con 0x7faf04069410 2022-11-27T15:14:32.566 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.544+0000 7faf09a75700 1 -- 172.21.15.85:0/4101372480 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf040cc310 con 0x7faf04069410 2022-11-27T15:14:32.566 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.544+0000 7faeebfff700 1 -- 172.21.15.85:0/4101372480 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7faef4010050 con 0x7faf04069410 2022-11-27T15:14:32.567 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.544+0000 7faeebfff700 1 -- 172.21.15.85:0/4101372480 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7faef4012760 con 0x7faf04069410 2022-11-27T15:14:32.567 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.544+0000 7faf09a75700 1 -- 172.21.15.85:0/4101372480 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0404e190 con 0x7faf04069410 2022-11-27T15:14:32.567 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.546+0000 7faeebfff700 1 -- 172.21.15.85:0/4101372480 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7faef40050d0 con 0x7faf04069410 2022-11-27T15:14:32.618 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:32.616+0000 7f6e4d15a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T15:14:32.629 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.627+0000 7faf09a75700 1 -- 172.21.15.85:0/4101372480 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "92e0fd37-a176-44da-847e-2bc153faa513", "id": 0} v 0) v1 -- 0x7faf0415d970 con 0x7faf04069410 2022-11-27T15:14:32.634 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.633+0000 7faeebfff700 1 -- 172.21.15.85:0/4101372480 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "92e0fd37-a176-44da-847e-2bc153faa513", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7faef4005340 con 0x7faf04069410 2022-11-27T15:14:32.636 INFO:teuthology.orchestra.run.smithi085.stdout:0 2022-11-27T15:14:32.638 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.634+0000 7faf09a75700 1 -- 172.21.15.85:0/4101372480 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7faf04069410 msgr2=0x7faf0414f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.638 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.634+0000 7faf09a75700 1 --2- 172.21.15.85:0/4101372480 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7faf04069410 0x7faf0414f7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faef4001ed0 tx=0x7faef40021e0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.639 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.634+0000 7faf09a75700 1 -- 172.21.15.85:0/4101372480 shutdown_connections 2022-11-27T15:14:32.639 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.634+0000 7faf09a75700 1 --2- 172.21.15.85:0/4101372480 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7faf0413f890 0x7faf040c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.640 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.634+0000 7faf09a75700 1 --2- 172.21.15.85:0/4101372480 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7faf04069410 0x7faf0414f7f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.640 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.634+0000 7faf09a75700 1 --2- 172.21.15.85:0/4101372480 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7faf0413f4c0 0x7faf040c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.641 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.634+0000 7faf09a75700 1 -- 172.21.15.85:0/4101372480 >> 172.21.15.85:0/4101372480 conn(0x7faf040cb2d0 msgr2=0x7faf040b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:32.641 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.634+0000 7faf09a75700 1 -- 172.21.15.85:0/4101372480 shutdown_connections 2022-11-27T15:14:32.641 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.634+0000 7faf09a75700 1 -- 172.21.15.85:0/4101372480 wait complete. 2022-11-27T15:14:32.647 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new e9828c4c-3cab-4b51-80fe-beaca70fd85a 1 2022-11-27T15:14:32.675 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:32.674+0000 7f2b58316200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T15:14:32.696 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:32.695+0000 7f6e4d15a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T15:14:32.753 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:32.752+0000 7f2b58316200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T15:14:32.775 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.766+0000 7f5422024700 1 Processor -- start 2022-11-27T15:14:32.775 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.766+0000 7f5422024700 1 -- start start 2022-11-27T15:14:32.776 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f5422024700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f541c0cd540 0x7f541c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:32.776 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f5422024700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0ce5f0 con 0x7f541c158630 2022-11-27T15:14:32.776 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f5422024700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0cc810 con 0x7f541c15cc20 2022-11-27T15:14:32.776 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f5422024700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541c0ce0f0 con 0x7f541c0cd540 2022-11-27T15:14:32.777 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f541affd700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f541c0cd540 0x7f541c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:32.777 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f541bfff700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f541c15cc20 0x7f541c0ca0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:57080/0 (socket says 172.21.15.85:57080) 2022-11-27T15:14:32.777 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f541bfff700 1 -- 172.21.15.85:0/2848119360 learned_addr learned my addr 172.21.15.85:0/2848119360 (peer_addr_for_me v1:172.21.15.85:0/0) 2022-11-27T15:14:32.777 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f541a7fc700 1 -- 172.21.15.85:0/2848119360 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303891301 0 0) 0x7f541c0cc810 con 0x7f541c15cc20 2022-11-27T15:14:32.777 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f541a7fc700 1 -- 172.21.15.85:0/2848119360 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54080035b0 con 0x7f541c15cc20 2022-11-27T15:14:32.778 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f541affd700 1 -- 172.21.15.85:0/2848119360 >> v1:172.21.15.85:6789/0 conn(0x7f541c15cc20 legacy=0x7f541c0ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.778 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f541a7fc700 1 -- 172.21.15.85:0/2848119360 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539786889 0 0) 0x7f541c0ce5f0 con 0x7f541c158630 2022-11-27T15:14:32.778 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f541affd700 1 -- 172.21.15.85:0/2848119360 >> v1:172.21.15.35:6789/0 conn(0x7f541c158630 legacy=0x7f541c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.778 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.767+0000 7f541affd700 1 -- 172.21.15.85:0/2848119360 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c0ce310 con 0x7f541c0cd540 2022-11-27T15:14:32.778 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.768+0000 7f541affd700 1 --2- 172.21.15.85:0/2848119360 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f541c0cd540 0x7f541c0b06c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5410009840 tx=0x7f5410009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee75e0fac036fea5 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:32.779 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.768+0000 7f541a7fc700 1 -- 172.21.15.85:0/2848119360 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410011830 con 0x7f541c0cd540 2022-11-27T15:14:32.779 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.768+0000 7f541a7fc700 1 -- 172.21.15.85:0/2848119360 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5410011990 con 0x7f541c0cd540 2022-11-27T15:14:32.779 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.768+0000 7f541a7fc700 1 -- 172.21.15.85:0/2848119360 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410011d20 con 0x7f541c0cd540 2022-11-27T15:14:32.780 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.768+0000 7f5422024700 1 -- 172.21.15.85:0/2848119360 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f541c0cd540 msgr2=0x7f541c0b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.780 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.768+0000 7f5422024700 1 --2- 172.21.15.85:0/2848119360 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f541c0cd540 0x7f541c0b06c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5410009840 tx=0x7f5410009b50 comp rx=0 tx=0).stop 2022-11-27T15:14:32.781 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.768+0000 7f5422024700 1 -- 172.21.15.85:0/2848119360 shutdown_connections 2022-11-27T15:14:32.781 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.768+0000 7f5422024700 1 --2- 172.21.15.85:0/2848119360 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f541c0cd540 0x7f541c0b06c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.781 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.768+0000 7f5422024700 1 -- 172.21.15.85:0/2848119360 >> 172.21.15.85:0/2848119360 conn(0x7f541c06d9c0 msgr2=0x7f541c0cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:32.782 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.768+0000 7f5422024700 1 -- 172.21.15.85:0/2848119360 shutdown_connections 2022-11-27T15:14:32.782 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.769+0000 7f5422024700 1 -- 172.21.15.85:0/2848119360 wait complete. 2022-11-27T15:14:32.782 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.770+0000 7f5422024700 1 Processor -- start 2022-11-27T15:14:32.782 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.770+0000 7f5422024700 1 -- start start 2022-11-27T15:14:32.783 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.770+0000 7f5422024700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f541c0cd540 0x7f541c0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:32.783 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.770+0000 7f5422024700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f541c158630 0x7f541c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:32.783 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.770+0000 7f5422024700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f541c15cc20 0x7f541c15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:32.783 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f5422024700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c158a00 con 0x7f541c158630 2022-11-27T15:14:32.784 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f5422024700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c158ca0 con 0x7f541c15cc20 2022-11-27T15:14:32.784 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f5422024700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c158f80 con 0x7f541c0cd540 2022-11-27T15:14:32.784 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541b7fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f541c0cd540 0x7f541c0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:32.784 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541bfff700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f541c15cc20 0x7f541c15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:32.785 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541b7fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f541c0cd540 0x7f541c0affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:36942/0 (socket says 172.21.15.85:36942) 2022-11-27T15:14:32.785 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541b7fe700 1 -- 172.21.15.85:0/2949514199 learned_addr learned my addr 172.21.15.85:0/2949514199 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:32.785 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541affd700 1 --2- 172.21.15.85:0/2949514199 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f541c158630 0x7f541c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:32.785 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541bfff700 1 -- 172.21.15.85:0/2949514199 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f541c0cd540 msgr2=0x7f541c0affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.786 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541bfff700 1 --2- 172.21.15.85:0/2949514199 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f541c0cd540 0x7f541c0affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.786 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541bfff700 1 -- 172.21.15.85:0/2949514199 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f541c158630 msgr2=0x7f541c159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.786 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541bfff700 1 --2- 172.21.15.85:0/2949514199 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f541c158630 0x7f541c159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.787 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541bfff700 1 -- 172.21.15.85:0/2949514199 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54100094a0 con 0x7f541c15cc20 2022-11-27T15:14:32.787 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541affd700 1 --2- 172.21.15.85:0/2949514199 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f541c158630 0x7f541c159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:32.787 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f541bfff700 1 --2- 172.21.15.85:0/2949514199 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f541c15cc20 0x7f541c15a280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f541400ca70 tx=0x7f541400cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:32.787 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f5403fff700 1 -- 172.21.15.85:0/2949514199 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400a9e0 con 0x7f541c15cc20 2022-11-27T15:14:32.788 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f5403fff700 1 -- 172.21.15.85:0/2949514199 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541400ab40 con 0x7f541c15cc20 2022-11-27T15:14:32.788 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f5403fff700 1 -- 172.21.15.85:0/2949514199 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414012400 con 0x7f541c15cc20 2022-11-27T15:14:32.788 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f5422024700 1 -- 172.21.15.85:0/2949514199 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c159260 con 0x7f541c15cc20 2022-11-27T15:14:32.788 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f5422024700 1 -- 172.21.15.85:0/2949514199 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c159820 con 0x7f541c15cc20 2022-11-27T15:14:32.788 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.771+0000 7f5403fff700 1 -- 172.21.15.85:0/2949514199 <== mon.1 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f5414019030 con 0x7f541c15cc20 2022-11-27T15:14:32.789 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.772+0000 7f5403fff700 1 -- 172.21.15.85:0/2949514199 <== mon.1 v2:172.21.15.85:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f541401c7f0 con 0x7f541c15cc20 2022-11-27T15:14:32.789 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.772+0000 7f5422024700 1 -- 172.21.15.85:0/2949514199 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c159a10 con 0x7f541c15cc20 2022-11-27T15:14:32.791 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.774+0000 7f5403fff700 1 -- 172.21.15.85:0/2949514199 <== mon.1 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f5414021060 con 0x7f541c15cc20 2022-11-27T15:14:32.828 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:32.827+0000 7f6e4d15a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T15:14:32.860 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.859+0000 7f5422024700 1 -- 172.21.15.85:0/2949514199 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e9828c4c-3cab-4b51-80fe-beaca70fd85a", "id": 1} v 0) v1 -- 0x7f541c155a20 con 0x7f541c15cc20 2022-11-27T15:14:32.867 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.866+0000 7f5403fff700 1 -- 172.21.15.85:0/2949514199 <== mon.1 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e9828c4c-3cab-4b51-80fe-beaca70fd85a", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5414017070 con 0x7f541c15cc20 2022-11-27T15:14:32.868 INFO:teuthology.orchestra.run.smithi085.stdout:1 2022-11-27T15:14:32.869 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.867+0000 7f5422024700 1 -- 172.21.15.85:0/2949514199 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f541c15cc20 msgr2=0x7f541c15a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:32.869 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.867+0000 7f5422024700 1 --2- 172.21.15.85:0/2949514199 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f541c15cc20 0x7f541c15a280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f541400ca70 tx=0x7f541400cd80 comp rx=0 tx=0).stop 2022-11-27T15:14:32.870 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.867+0000 7f5422024700 1 -- 172.21.15.85:0/2949514199 shutdown_connections 2022-11-27T15:14:32.870 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.867+0000 7f5422024700 1 --2- 172.21.15.85:0/2949514199 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f541c15cc20 0x7f541c15a280 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.870 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.867+0000 7f5422024700 1 --2- 172.21.15.85:0/2949514199 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f541c0cd540 0x7f541c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.870 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.867+0000 7f5422024700 1 --2- 172.21.15.85:0/2949514199 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f541c158630 0x7f541c159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:32.871 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.867+0000 7f5422024700 1 -- 172.21.15.85:0/2949514199 >> 172.21.15.85:0/2949514199 conn(0x7f541c06d9c0 msgr2=0x7f541c154930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:32.871 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.867+0000 7f5422024700 1 -- 172.21.15.85:0/2949514199 shutdown_connections 2022-11-27T15:14:32.871 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:32.867+0000 7f5422024700 1 -- 172.21.15.85:0/2949514199 wait complete. 2022-11-27T15:14:32.880 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 80ba487d-307a-46d3-bf27-fb037321414c 2 2022-11-27T15:14:32.907 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:32.907+0000 7f2b58316200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T15:14:32.914 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:32.913+0000 7f6e4d15a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T15:14:32.991 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:32.990+0000 7f2b58316200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T15:14:33.001 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.000+0000 7f19f098f700 1 Processor -- start 2022-11-27T15:14:33.002 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.000+0000 7f19f098f700 1 -- start start 2022-11-27T15:14:33.014 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19f098f700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f19ec0b1ed0 0x7f19ec15a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.014 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19f098f700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f19ec0ccbf0 0x7f19ec0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.014 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19f098f700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0ca750 con 0x7f19ec156fb0 2022-11-27T15:14:33.014 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19f098f700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec15ab40 con 0x7f19ec0b1ed0 2022-11-27T15:14:33.015 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19f098f700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec0cc110 con 0x7f19ec0ccbf0 2022-11-27T15:14:33.015 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19ea59c700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f19ec156fb0 0x7f19ec0b17c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:57088/0 (socket says 172.21.15.85:57088) 2022-11-27T15:14:33.015 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19ea59c700 1 -- 172.21.15.85:0/970936665 learned_addr learned my addr 172.21.15.85:0/970936665 (peer_addr_for_me v1:172.21.15.85:0/0) 2022-11-27T15:14:33.016 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19ead9d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f19ec0ccbf0 0x7f19ec0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.016 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19e9d9b700 1 --2- 172.21.15.85:0/970936665 >> v2:172.21.15.35:3300/0 conn(0x7f19ec0b1ed0 0x7f19ec15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.016 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19e959a700 1 -- 172.21.15.85:0/970936665 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483519862 0 0) 0x7f19ec0ca750 con 0x7f19ec156fb0 2022-11-27T15:14:33.016 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19e959a700 1 -- 172.21.15.85:0/970936665 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19d00035b0 con 0x7f19ec156fb0 2022-11-27T15:14:33.017 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19ead9d700 1 -- 172.21.15.85:0/970936665 >> v2:172.21.15.35:3300/0 conn(0x7f19ec0b1ed0 msgr2=0x7f19ec15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:33.017 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19ead9d700 1 --2- 172.21.15.85:0/970936665 >> v2:172.21.15.35:3300/0 conn(0x7f19ec0b1ed0 0x7f19ec15a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.017 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19ead9d700 1 -- 172.21.15.85:0/970936665 >> v1:172.21.15.85:6789/0 conn(0x7f19ec156fb0 legacy=0x7f19ec0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.017 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.001+0000 7f19ead9d700 1 -- 172.21.15.85:0/970936665 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0cc3f0 con 0x7f19ec0ccbf0 2022-11-27T15:14:33.018 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19ead9d700 1 --2- 172.21.15.85:0/970936665 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec0ccbf0 0x7f19ec0cd910 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f19ec159b80 tx=0x7f19e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a6f9a0ac0178b8b server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:33.018 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19e959a700 1 -- 172.21.15.85:0/970936665 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e400e070 con 0x7f19ec0ccbf0 2022-11-27T15:14:33.018 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19e959a700 1 -- 172.21.15.85:0/970936665 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e400bb20 con 0x7f19ec0ccbf0 2022-11-27T15:14:33.018 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19e959a700 1 -- 172.21.15.85:0/970936665 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4012430 con 0x7f19ec0ccbf0 2022-11-27T15:14:33.019 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19f098f700 1 -- 172.21.15.85:0/970936665 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec0ccbf0 msgr2=0x7f19ec0cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.019 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19f098f700 1 --2- 172.21.15.85:0/970936665 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec0ccbf0 0x7f19ec0cd910 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f19ec159b80 tx=0x7f19e400b330 comp rx=0 tx=0).stop 2022-11-27T15:14:33.019 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19f098f700 1 -- 172.21.15.85:0/970936665 shutdown_connections 2022-11-27T15:14:33.020 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19f098f700 1 --2- 172.21.15.85:0/970936665 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec0ccbf0 0x7f19ec0cd910 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.020 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19f098f700 1 --2- 172.21.15.85:0/970936665 >> v2:172.21.15.35:3300/0 conn(0x7f19ec0b1ed0 0x7f19ec15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.020 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19f098f700 1 -- 172.21.15.85:0/970936665 >> 172.21.15.85:0/970936665 conn(0x7f19ec06de90 msgr2=0x7f19ec06e260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:33.021 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19f098f700 1 -- 172.21.15.85:0/970936665 shutdown_connections 2022-11-27T15:14:33.021 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.002+0000 7f19f098f700 1 -- 172.21.15.85:0/970936665 wait complete. 2022-11-27T15:14:33.022 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.004+0000 7f19f098f700 1 Processor -- start 2022-11-27T15:14:33.022 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.004+0000 7f19f098f700 1 -- start start 2022-11-27T15:14:33.023 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19f098f700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f19ec0b1ed0 0x7f19ec14b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.023 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19f098f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f19ec0ccbf0 0x7f19ec15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.024 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19f098f700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec156fb0 0x7f19ec0b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.024 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19f098f700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0c4520 con 0x7f19ec0b1ed0 2022-11-27T15:14:33.024 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19f098f700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0c3910 con 0x7f19ec156fb0 2022-11-27T15:14:33.025 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19f098f700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0c3bf0 con 0x7f19ec0ccbf0 2022-11-27T15:14:33.025 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19e9d9b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f19ec0ccbf0 0x7f19ec15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.026 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19ead9d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec156fb0 0x7f19ec0b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.026 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19ead9d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec156fb0 0x7f19ec0b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:39072/0 (socket says 172.21.15.85:39072) 2022-11-27T15:14:33.027 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19e9d9b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f19ec0ccbf0 0x7f19ec15c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:36956/0 (socket says 172.21.15.85:36956) 2022-11-27T15:14:33.027 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19ead9d700 1 -- 172.21.15.85:0/1552042823 learned_addr learned my addr 172.21.15.85:0/1552042823 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:33.028 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19ea59c700 1 --2- 172.21.15.85:0/1552042823 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f19ec0b1ed0 0x7f19ec14b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.028 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19ead9d700 1 -- 172.21.15.85:0/1552042823 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f19ec0ccbf0 msgr2=0x7f19ec15c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.029 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19ead9d700 1 --2- 172.21.15.85:0/1552042823 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f19ec0ccbf0 0x7f19ec15c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.029 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19ead9d700 1 -- 172.21.15.85:0/1552042823 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f19ec0b1ed0 msgr2=0x7f19ec14b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.030 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19ead9d700 1 --2- 172.21.15.85:0/1552042823 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f19ec0b1ed0 0x7f19ec14b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.030 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19ead9d700 1 -- 172.21.15.85:0/1552042823 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4009a70 con 0x7f19ec156fb0 2022-11-27T15:14:33.030 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19ead9d700 1 --2- 172.21.15.85:0/1552042823 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec156fb0 0x7f19ec0b33a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f19e4014040 tx=0x7f19e4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:33.031 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19daffd700 1 -- 172.21.15.85:0/1552042823 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4012430 con 0x7f19ec156fb0 2022-11-27T15:14:33.031 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19daffd700 1 -- 172.21.15.85:0/1552042823 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e4020070 con 0x7f19ec156fb0 2022-11-27T15:14:33.032 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19f098f700 1 -- 172.21.15.85:0/1552042823 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec0c3ed0 con 0x7f19ec156fb0 2022-11-27T15:14:33.032 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19daffd700 1 -- 172.21.15.85:0/1552042823 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e40098f0 con 0x7f19ec156fb0 2022-11-27T15:14:33.032 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.005+0000 7f19f098f700 1 -- 172.21.15.85:0/1552042823 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec14f310 con 0x7f19ec156fb0 2022-11-27T15:14:33.033 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.006+0000 7f19daffd700 1 -- 172.21.15.85:0/1552042823 <== mon.1 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f19e4004330 con 0x7f19ec156fb0 2022-11-27T15:14:33.033 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.006+0000 7f19daffd700 1 -- 172.21.15.85:0/1552042823 <== mon.1 v2:172.21.15.85:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f19e4022070 con 0x7f19ec156fb0 2022-11-27T15:14:33.033 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.006+0000 7f19f098f700 1 -- 172.21.15.85:0/1552042823 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec04e190 con 0x7f19ec156fb0 2022-11-27T15:14:33.033 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.008+0000 7f19daffd700 1 -- 172.21.15.85:0/1552042823 <== mon.1 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f19e40045c0 con 0x7f19ec156fb0 2022-11-27T15:14:33.093 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.091+0000 7f19f098f700 1 -- 172.21.15.85:0/1552042823 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "80ba487d-307a-46d3-bf27-fb037321414c", "id": 2} v 0) v1 -- 0x7f19ec0cc6d0 con 0x7f19ec156fb0 2022-11-27T15:14:33.100 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.098+0000 7f19daffd700 1 -- 172.21.15.85:0/1552042823 <== mon.1 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "80ba487d-307a-46d3-bf27-fb037321414c", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f19e4004760 con 0x7f19ec156fb0 2022-11-27T15:14:33.100 INFO:teuthology.orchestra.run.smithi085.stdout:2 2022-11-27T15:14:33.102 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.100+0000 7f19f098f700 1 -- 172.21.15.85:0/1552042823 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec156fb0 msgr2=0x7f19ec0b33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.102 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.100+0000 7f19f098f700 1 --2- 172.21.15.85:0/1552042823 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec156fb0 0x7f19ec0b33a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f19e4014040 tx=0x7f19e4004ad0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.102 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.100+0000 7f19f098f700 1 -- 172.21.15.85:0/1552042823 shutdown_connections 2022-11-27T15:14:33.102 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.100+0000 7f19f098f700 1 --2- 172.21.15.85:0/1552042823 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f19ec156fb0 0x7f19ec0b33a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.103 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.100+0000 7f19f098f700 1 --2- 172.21.15.85:0/1552042823 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f19ec0b1ed0 0x7f19ec14b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.103 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.100+0000 7f19f098f700 1 --2- 172.21.15.85:0/1552042823 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f19ec0ccbf0 0x7f19ec15c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.103 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.100+0000 7f19f098f700 1 -- 172.21.15.85:0/1552042823 >> 172.21.15.85:0/1552042823 conn(0x7f19ec06de90 msgr2=0x7f19ec0b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:33.103 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.100+0000 7f19f098f700 1 -- 172.21.15.85:0/1552042823 shutdown_connections 2022-11-27T15:14:33.104 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.100+0000 7f19f098f700 1 -- 172.21.15.85:0/1552042823 wait complete. 2022-11-27T15:14:33.113 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 962bb8b8-e6be-48ce-99c8-4697ba5fd364 3 2022-11-27T15:14:33.272 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.271+0000 7f4977232700 1 Processor -- start 2022-11-27T15:14:33.272 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.271+0000 7f4977232700 1 -- start start 2022-11-27T15:14:33.282 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.271+0000 7f4977232700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f4970155d90 0x7f49700cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.284 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.271+0000 7f4977232700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497006a930 con 0x7f49700b0dd0 2022-11-27T15:14:33.285 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.271+0000 7f4977232700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497015bee0 con 0x7f49700ccd50 2022-11-27T15:14:33.289 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.271+0000 7f4977232700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497015c1c0 con 0x7f4970155d90 2022-11-27T15:14:33.290 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f496ffff700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f49700ccd50 0x7f49700b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:57102/0 (socket says 172.21.15.85:57102) 2022-11-27T15:14:33.290 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f496ffff700 1 -- 172.21.15.85:0/2036732040 learned_addr learned my addr 172.21.15.85:0/2036732040 (peer_addr_for_me v1:172.21.15.85:0/0) 2022-11-27T15:14:33.291 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f4974fce700 1 --2- 172.21.15.85:0/2036732040 >> v2:172.21.15.35:3300/0 conn(0x7f4970155d90 0x7f49700cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.291 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f496f7fe700 1 -- 172.21.15.85:0/2036732040 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924812337 0 0) 0x7f497015bee0 con 0x7f49700ccd50 2022-11-27T15:14:33.292 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f496f7fe700 1 -- 172.21.15.85:0/2036732040 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49540035b0 con 0x7f49700ccd50 2022-11-27T15:14:33.292 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f4974fce700 1 -- 172.21.15.85:0/2036732040 >> v1:172.21.15.85:6789/0 conn(0x7f49700ccd50 legacy=0x7f49700b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.292 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f4974fce700 1 -- 172.21.15.85:0/2036732040 >> v1:172.21.15.35:6789/0 conn(0x7f49700b0dd0 legacy=0x7f497015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.292 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f4974fce700 1 -- 172.21.15.85:0/2036732040 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497015c4a0 con 0x7f4970155d90 2022-11-27T15:14:33.293 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f4974fce700 1 --2- 172.21.15.85:0/2036732040 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f4970155d90 0x7f49700cc810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4964009820 tx=0x7f4964009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a281053ffefdf861 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:33.293 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f496f7fe700 1 -- 172.21.15.85:0/2036732040 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964005e30 con 0x7f4970155d90 2022-11-27T15:14:33.294 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f496f7fe700 1 -- 172.21.15.85:0/2036732040 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4964006030 con 0x7f4970155d90 2022-11-27T15:14:33.295 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.272+0000 7f496f7fe700 1 -- 172.21.15.85:0/2036732040 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49640063c0 con 0x7f4970155d90 2022-11-27T15:14:33.296 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.273+0000 7f4977232700 1 -- 172.21.15.85:0/2036732040 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f4970155d90 msgr2=0x7f49700cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.296 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.273+0000 7f4977232700 1 --2- 172.21.15.85:0/2036732040 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f4970155d90 0x7f49700cc810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4964009820 tx=0x7f4964009b30 comp rx=0 tx=0).stop 2022-11-27T15:14:33.297 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.273+0000 7f4977232700 1 -- 172.21.15.85:0/2036732040 shutdown_connections 2022-11-27T15:14:33.298 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.273+0000 7f4977232700 1 --2- 172.21.15.85:0/2036732040 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f4970155d90 0x7f49700cc810 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.298 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.273+0000 7f4977232700 1 -- 172.21.15.85:0/2036732040 >> 172.21.15.85:0/2036732040 conn(0x7f497006d9c0 msgr2=0x7f49700cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:33.299 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.273+0000 7f4977232700 1 -- 172.21.15.85:0/2036732040 shutdown_connections 2022-11-27T15:14:33.299 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.273+0000 7f4977232700 1 -- 172.21.15.85:0/2036732040 wait complete. 2022-11-27T15:14:33.301 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.274+0000 7f4977232700 1 Processor -- start 2022-11-27T15:14:33.301 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.274+0000 7f4977232700 1 -- start start 2022-11-27T15:14:33.301 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f4977232700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49700b0dd0 0x7f49700b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.302 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f4977232700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49700ccd50 0x7f49700aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.304 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f4977232700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f4970155d90 0x7f4970145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.304 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f4977232700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497014f210 con 0x7f4970155d90 2022-11-27T15:14:33.304 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f4977232700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700ad910 con 0x7f49700b0dd0 2022-11-27T15:14:33.305 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f4977232700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49700aadf0 con 0x7f49700ccd50 2022-11-27T15:14:33.305 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f4974fce700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49700b0dd0 0x7f49700b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.305 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f496ffff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49700ccd50 0x7f49700aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.306 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f4974fce700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49700b0dd0 0x7f49700b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:39080/0 (socket says 172.21.15.85:39080) 2022-11-27T15:14:33.345 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f4974fce700 1 -- 172.21.15.85:0/2190240151 learned_addr learned my addr 172.21.15.85:0/2190240151 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:33.346 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f49757cf700 1 --2- 172.21.15.85:0/2190240151 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f4970155d90 0x7f4970145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.346 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.275+0000 7f496ffff700 1 -- 172.21.15.85:0/2190240151 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49700b0dd0 msgr2=0x7f49700b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.347 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f496ffff700 1 --2- 172.21.15.85:0/2190240151 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49700b0dd0 0x7f49700b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.347 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f496ffff700 1 -- 172.21.15.85:0/2190240151 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f4970155d90 msgr2=0x7f4970145e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.347 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f496ffff700 1 --2- 172.21.15.85:0/2190240151 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f4970155d90 0x7f4970145e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.348 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f496ffff700 1 -- 172.21.15.85:0/2190240151 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4964009430 con 0x7f49700ccd50 2022-11-27T15:14:33.348 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f49757cf700 1 --2- 172.21.15.85:0/2190240151 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f4970155d90 0x7f4970145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:33.348 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f4974fce700 1 --2- 172.21.15.85:0/2190240151 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49700b0dd0 0x7f49700b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:33.348 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f496ffff700 1 --2- 172.21.15.85:0/2190240151 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49700ccd50 0x7f49700aa8b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f495c00c4c0 tx=0x7f495c00c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:33.349 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f496d7fa700 1 -- 172.21.15.85:0/2190240151 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c005d70 con 0x7f49700ccd50 2022-11-27T15:14:33.349 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f496d7fa700 1 -- 172.21.15.85:0/2190240151 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495c00ce30 con 0x7f49700ccd50 2022-11-27T15:14:33.349 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f4977232700 1 -- 172.21.15.85:0/2190240151 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49701463d0 con 0x7f49700ccd50 2022-11-27T15:14:33.349 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f496d7fa700 1 -- 172.21.15.85:0/2190240151 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c0072f0 con 0x7f49700ccd50 2022-11-27T15:14:33.350 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f4977232700 1 -- 172.21.15.85:0/2190240151 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49700ad2c0 con 0x7f49700ccd50 2022-11-27T15:14:33.350 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f496d7fa700 1 -- 172.21.15.85:0/2190240151 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f495c019030 con 0x7f49700ccd50 2022-11-27T15:14:33.350 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.276+0000 7f496d7fa700 1 -- 172.21.15.85:0/2190240151 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f495c01b840 con 0x7f49700ccd50 2022-11-27T15:14:33.351 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.277+0000 7f4977232700 1 -- 172.21.15.85:0/2190240151 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49700659f0 con 0x7f49700ccd50 2022-11-27T15:14:33.351 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.280+0000 7f496d7fa700 1 -- 172.21.15.85:0/2190240151 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f495c017070 con 0x7f49700ccd50 2022-11-27T15:14:33.364 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.362+0000 7f4977232700 1 -- 172.21.15.85:0/2190240151 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "962bb8b8-e6be-48ce-99c8-4697ba5fd364", "id": 3} v 0) v1 -- 0x7f497004e190 con 0x7f49700ccd50 2022-11-27T15:14:33.371 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f6e4d15a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T15:14:33.372 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f496d7fa700 1 -- 172.21.15.85:0/2190240151 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "962bb8b8-e6be-48ce-99c8-4697ba5fd364", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f495c0074f0 con 0x7f49700ccd50 2022-11-27T15:14:33.373 INFO:teuthology.orchestra.run.smithi085.stdout:3 2022-11-27T15:14:33.375 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f4977232700 1 -- 172.21.15.85:0/2190240151 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49700ccd50 msgr2=0x7f49700aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.376 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f4977232700 1 --2- 172.21.15.85:0/2190240151 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49700ccd50 0x7f49700aa8b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f495c00c4c0 tx=0x7f495c00c7d0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.376 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f4977232700 1 -- 172.21.15.85:0/2190240151 shutdown_connections 2022-11-27T15:14:33.377 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f4977232700 1 --2- 172.21.15.85:0/2190240151 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f4970155d90 0x7f4970145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.377 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f4977232700 1 --2- 172.21.15.85:0/2190240151 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49700b0dd0 0x7f49700b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.378 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f4977232700 1 --2- 172.21.15.85:0/2190240151 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49700ccd50 0x7f49700aa8b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.378 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f4977232700 1 -- 172.21.15.85:0/2190240151 >> 172.21.15.85:0/2190240151 conn(0x7f497006d9c0 msgr2=0x7f497015cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:33.378 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f4977232700 1 -- 172.21.15.85:0/2190240151 shutdown_connections 2022-11-27T15:14:33.379 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.370+0000 7f4977232700 1 -- 172.21.15.85:0/2190240151 wait complete. 2022-11-27T15:14:33.383 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new fd1b4ee8-0942-467c-b3cc-990bfb7c34c7 4 2022-11-27T15:14:33.427 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:33.425+0000 7f2b58316200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T15:14:33.508 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.507+0000 7ff3dc443700 1 Processor -- start 2022-11-27T15:14:33.509 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.507+0000 7ff3dc443700 1 -- start start 2022-11-27T15:14:33.515 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3dc443700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7ff3d41572d0 0x7ff3d406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.515 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3dc443700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7ff3d40c90f0 0x7ff3d4159e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.516 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3dc443700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d415d4a0 con 0x7ff3d4070e20 2022-11-27T15:14:33.516 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3dc443700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d406dfa0 con 0x7ff3d41572d0 2022-11-27T15:14:33.516 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3dc443700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d415d780 con 0x7ff3d40c90f0 2022-11-27T15:14:33.516 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3da9e0700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7ff3d40c90f0 0x7ff3d4159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.517 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3da9e0700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7ff3d40c90f0 0x7ff3d4159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:39088/0 (socket says 172.21.15.85:39088) 2022-11-27T15:14:33.517 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3da9e0700 1 -- 172.21.15.85:0/1050927556 learned_addr learned my addr 172.21.15.85:0/1050927556 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:33.517 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3da1df700 1 --2- 172.21.15.85:0/1050927556 >> v2:172.21.15.35:3300/0 conn(0x7ff3d41572d0 0x7ff3d406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.517 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3da9e0700 1 -- 172.21.15.85:0/1050927556 >> v2:172.21.15.35:3300/0 conn(0x7ff3d41572d0 msgr2=0x7ff3d406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:33.517 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3da9e0700 1 --2- 172.21.15.85:0/1050927556 >> v2:172.21.15.35:3300/0 conn(0x7ff3d41572d0 0x7ff3d406da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.518 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3da9e0700 1 -- 172.21.15.85:0/1050927556 >> v1:172.21.15.35:6789/0 conn(0x7ff3d4070e20 legacy=0x7ff3d40c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.518 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3da9e0700 1 -- 172.21.15.85:0/1050927556 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d415da60 con 0x7ff3d40c90f0 2022-11-27T15:14:33.518 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.508+0000 7ff3da9e0700 1 --2- 172.21.15.85:0/1050927556 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d40c90f0 0x7ff3d4159e90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40addb0 tx=0x7ff3cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f4327c9e6c84d4c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:33.518 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3d91dd700 1 -- 172.21.15.85:0/1050927556 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00e070 con 0x7ff3d40c90f0 2022-11-27T15:14:33.518 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3d91dd700 1 -- 172.21.15.85:0/1050927556 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3cc00b910 con 0x7ff3d40c90f0 2022-11-27T15:14:33.519 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3d91dd700 1 -- 172.21.15.85:0/1050927556 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00bcc0 con 0x7ff3d40c90f0 2022-11-27T15:14:33.519 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3dc443700 1 -- 172.21.15.85:0/1050927556 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d40c90f0 msgr2=0x7ff3d4159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.519 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3dc443700 1 --2- 172.21.15.85:0/1050927556 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d40c90f0 0x7ff3d4159e90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40addb0 tx=0x7ff3cc00b330 comp rx=0 tx=0).stop 2022-11-27T15:14:33.519 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3dc443700 1 -- 172.21.15.85:0/1050927556 shutdown_connections 2022-11-27T15:14:33.520 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3dc443700 1 --2- 172.21.15.85:0/1050927556 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d40c90f0 0x7ff3d4159e90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.520 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3dc443700 1 --2- 172.21.15.85:0/1050927556 >> v2:172.21.15.35:3300/0 conn(0x7ff3d41572d0 0x7ff3d406da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.520 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3dc443700 1 -- 172.21.15.85:0/1050927556 >> 172.21.15.85:0/1050927556 conn(0x7ff3d40aff10 msgr2=0x7ff3d40b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:33.520 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3dc443700 1 -- 172.21.15.85:0/1050927556 shutdown_connections 2022-11-27T15:14:33.521 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.509+0000 7ff3dc443700 1 -- 172.21.15.85:0/1050927556 wait complete. 2022-11-27T15:14:33.521 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3dc443700 1 Processor -- start 2022-11-27T15:14:33.521 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3dc443700 1 -- start start 2022-11-27T15:14:33.522 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3dc443700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d4070e20 0x7ff3d40b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.522 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3dc443700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff3d40c90f0 0x7ff3d4155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.522 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3dc443700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff3d41572d0 0x7ff3d40c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.522 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3dc443700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40b2e50 con 0x7ff3d41572d0 2022-11-27T15:14:33.522 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3dc443700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40b32e0 con 0x7ff3d4070e20 2022-11-27T15:14:33.523 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3dc443700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4156200 con 0x7ff3d40c90f0 2022-11-27T15:14:33.523 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3da1df700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d4070e20 0x7ff3d40b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.523 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3da1df700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d4070e20 0x7ff3d40b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:39090/0 (socket says 172.21.15.85:39090) 2022-11-27T15:14:33.524 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3da1df700 1 -- 172.21.15.85:0/2009051757 learned_addr learned my addr 172.21.15.85:0/2009051757 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:33.524 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3d99de700 1 --2- 172.21.15.85:0/2009051757 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff3d40c90f0 0x7ff3d4155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.524 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.511+0000 7ff3da9e0700 1 --2- 172.21.15.85:0/2009051757 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff3d41572d0 0x7ff3d40c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.524 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3da1df700 1 -- 172.21.15.85:0/2009051757 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff3d40c90f0 msgr2=0x7ff3d4155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.525 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3da1df700 1 --2- 172.21.15.85:0/2009051757 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff3d40c90f0 0x7ff3d4155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.525 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3da1df700 1 -- 172.21.15.85:0/2009051757 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff3d41572d0 msgr2=0x7ff3d40c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.525 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3da1df700 1 --2- 172.21.15.85:0/2009051757 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff3d41572d0 0x7ff3d40c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.525 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3da1df700 1 -- 172.21.15.85:0/2009051757 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc009b30 con 0x7ff3d4070e20 2022-11-27T15:14:33.526 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3da1df700 1 --2- 172.21.15.85:0/2009051757 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d4070e20 0x7ff3d40b1c10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000e870 tx=0x7ff3d000eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:33.526 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3c2ffd700 1 -- 172.21.15.85:0/2009051757 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0009590 con 0x7ff3d4070e20 2022-11-27T15:14:33.526 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3c2ffd700 1 -- 172.21.15.85:0/2009051757 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d0013680 con 0x7ff3d4070e20 2022-11-27T15:14:33.526 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3dc443700 1 -- 172.21.15.85:0/2009051757 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d40b2240 con 0x7ff3d4070e20 2022-11-27T15:14:33.527 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3c2ffd700 1 -- 172.21.15.85:0/2009051757 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00139f0 con 0x7ff3d4070e20 2022-11-27T15:14:33.527 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3dc443700 1 -- 172.21.15.85:0/2009051757 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d40b2800 con 0x7ff3d4070e20 2022-11-27T15:14:33.527 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3c2ffd700 1 -- 172.21.15.85:0/2009051757 <== mon.1 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff3d0023070 con 0x7ff3d4070e20 2022-11-27T15:14:33.527 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.512+0000 7ff3c2ffd700 1 -- 172.21.15.85:0/2009051757 <== mon.1 v2:172.21.15.85:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7ff3d00128d0 con 0x7ff3d4070e20 2022-11-27T15:14:33.527 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.513+0000 7ff3dc443700 1 -- 172.21.15.85:0/2009051757 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d40b3130 con 0x7ff3d4070e20 2022-11-27T15:14:33.528 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.515+0000 7ff3c2ffd700 1 -- 172.21.15.85:0/2009051757 <== mon.1 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7ff3d0021060 con 0x7ff3d4070e20 2022-11-27T15:14:33.599 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.598+0000 7ff3dc443700 1 -- 172.21.15.85:0/2009051757 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fd1b4ee8-0942-467c-b3cc-990bfb7c34c7", "id": 4} v 0) v1 -- 0x7ff3d406aed0 con 0x7ff3d4070e20 2022-11-27T15:14:33.606 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.605+0000 7ff3c2ffd700 1 -- 172.21.15.85:0/2009051757 <== mon.1 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fd1b4ee8-0942-467c-b3cc-990bfb7c34c7", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff3d002a0b0 con 0x7ff3d4070e20 2022-11-27T15:14:33.607 INFO:teuthology.orchestra.run.smithi085.stdout:4 2022-11-27T15:14:33.608 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.606+0000 7ff3dc443700 1 -- 172.21.15.85:0/2009051757 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d4070e20 msgr2=0x7ff3d40b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.609 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.606+0000 7ff3dc443700 1 --2- 172.21.15.85:0/2009051757 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d4070e20 0x7ff3d40b1c10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000e870 tx=0x7ff3d000eb80 comp rx=0 tx=0).stop 2022-11-27T15:14:33.609 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.606+0000 7ff3dc443700 1 -- 172.21.15.85:0/2009051757 shutdown_connections 2022-11-27T15:14:33.609 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.606+0000 7ff3dc443700 1 --2- 172.21.15.85:0/2009051757 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff3d41572d0 0x7ff3d40c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.610 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.606+0000 7ff3dc443700 1 --2- 172.21.15.85:0/2009051757 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff3d4070e20 0x7ff3d40b1c10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.610 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.606+0000 7ff3dc443700 1 --2- 172.21.15.85:0/2009051757 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff3d40c90f0 0x7ff3d4155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.610 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.606+0000 7ff3dc443700 1 -- 172.21.15.85:0/2009051757 >> 172.21.15.85:0/2009051757 conn(0x7ff3d40aff10 msgr2=0x7ff3d4069ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:33.610 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.606+0000 7ff3dc443700 1 -- 172.21.15.85:0/2009051757 shutdown_connections 2022-11-27T15:14:33.610 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.606+0000 7ff3dc443700 1 -- 172.21.15.85:0/2009051757 wait complete. 2022-11-27T15:14:33.619 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new f6133163-76e1-4748-83ad-e32cfaaaaf41 5 2022-11-27T15:14:33.739 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.738+0000 7fd08bd64700 1 Processor -- start 2022-11-27T15:14:33.740 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.738+0000 7fd08bd64700 1 -- start start 2022-11-27T15:14:33.746 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd08bd64700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd08415b850 0x7fd0840cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.746 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd08bd64700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415fdb0 0x7fd0840cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.746 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd08bd64700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08415bc20 con 0x7fd0840cca40 2022-11-27T15:14:33.747 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd08bd64700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0840ccec0 con 0x7fd08415b850 2022-11-27T15:14:33.747 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd08bd64700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840cdef0 con 0x7fd08415fdb0 2022-11-27T15:14:33.747 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd089b00700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd08415b850 0x7fd0840cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.748 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd0892ff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415fdb0 0x7fd0840cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.748 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd089b00700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd08415b850 0x7fd0840cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:39096/0 (socket says 172.21.15.85:39096) 2022-11-27T15:14:33.748 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd0892ff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415fdb0 0x7fd0840cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:36982/0 (socket says 172.21.15.85:36982) 2022-11-27T15:14:33.748 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd089b00700 1 -- 172.21.15.85:0/38490495 learned_addr learned my addr 172.21.15.85:0/38490495 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:33.749 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd089b00700 1 -- 172.21.15.85:0/38490495 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415fdb0 msgr2=0x7fd0840cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:33.749 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd089b00700 1 --2- 172.21.15.85:0/38490495 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415fdb0 0x7fd0840cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.749 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd089b00700 1 -- 172.21.15.85:0/38490495 >> v1:172.21.15.35:6789/0 conn(0x7fd0840cca40 legacy=0x7fd0840b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.749 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.739+0000 7fd089b00700 1 -- 172.21.15.85:0/38490495 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0840ca430 con 0x7fd08415b850 2022-11-27T15:14:33.749 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd089b00700 1 --2- 172.21.15.85:0/38490495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd08415b850 0x7fd0840cd910 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd0800098f0 tx=0x7fd080009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61596b6465cee301 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:33.750 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd088afe700 1 -- 172.21.15.85:0/38490495 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080011830 con 0x7fd08415b850 2022-11-27T15:14:33.750 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd088afe700 1 -- 172.21.15.85:0/38490495 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd080011990 con 0x7fd08415b850 2022-11-27T15:14:33.750 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd088afe700 1 -- 172.21.15.85:0/38490495 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080011d20 con 0x7fd08415b850 2022-11-27T15:14:33.750 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd08bd64700 1 -- 172.21.15.85:0/38490495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd08415b850 msgr2=0x7fd0840cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.751 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd08bd64700 1 --2- 172.21.15.85:0/38490495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd08415b850 0x7fd0840cd910 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd0800098f0 tx=0x7fd080009c00 comp rx=0 tx=0).stop 2022-11-27T15:14:33.751 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd08bd64700 1 -- 172.21.15.85:0/38490495 shutdown_connections 2022-11-27T15:14:33.751 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd08bd64700 1 --2- 172.21.15.85:0/38490495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd08415b850 0x7fd0840cd910 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.751 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd08bd64700 1 --2- 172.21.15.85:0/38490495 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415fdb0 0x7fd0840cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.751 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd08bd64700 1 -- 172.21.15.85:0/38490495 >> 172.21.15.85:0/38490495 conn(0x7fd08406de90 msgr2=0x7fd08406e260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:33.752 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd08bd64700 1 -- 172.21.15.85:0/38490495 shutdown_connections 2022-11-27T15:14:33.752 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.740+0000 7fd08bd64700 1 -- 172.21.15.85:0/38490495 wait complete. 2022-11-27T15:14:33.752 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd08bd64700 1 Processor -- start 2022-11-27T15:14:33.752 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd08bd64700 1 -- start start 2022-11-27T15:14:33.753 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd08bd64700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0840cca40 0x7fd0840c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.753 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd08bd64700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415b850 0x7fd0840c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.753 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd08bd64700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fd08415fdb0 0x7fd084152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:33.753 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd08bd64700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840c4810 con 0x7fd08415fdb0 2022-11-27T15:14:33.754 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd08bd64700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840c6c10 con 0x7fd0840cca40 2022-11-27T15:14:33.754 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd08bd64700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840ca020 con 0x7fd08415b850 2022-11-27T15:14:33.754 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd089b00700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0840cca40 0x7fd0840c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.754 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd089b00700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0840cca40 0x7fd0840c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:39102/0 (socket says 172.21.15.85:39102) 2022-11-27T15:14:33.755 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd089b00700 1 -- 172.21.15.85:0/2480364391 learned_addr learned my addr 172.21.15.85:0/2480364391 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:33.755 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.742+0000 7fd0892ff700 1 --2- 172.21.15.85:0/2480364391 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415b850 0x7fd0840c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.755 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd08a301700 1 --2- 172.21.15.85:0/2480364391 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fd08415fdb0 0x7fd084152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:33.755 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0892ff700 1 -- 172.21.15.85:0/2480364391 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0840cca40 msgr2=0x7fd0840c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.756 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0892ff700 1 --2- 172.21.15.85:0/2480364391 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0840cca40 0x7fd0840c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.756 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0892ff700 1 -- 172.21.15.85:0/2480364391 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fd08415fdb0 msgr2=0x7fd084152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.756 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0892ff700 1 --2- 172.21.15.85:0/2480364391 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fd08415fdb0 0x7fd084152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.756 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0892ff700 1 -- 172.21.15.85:0/2480364391 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd080009550 con 0x7fd08415b850 2022-11-27T15:14:33.757 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0892ff700 1 --2- 172.21.15.85:0/2480364391 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415b850 0x7fd0840c66d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd07400e930 tx=0x7fd07400ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:33.757 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0727fc700 1 -- 172.21.15.85:0/2480364391 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074013680 con 0x7fd08415b850 2022-11-27T15:14:33.757 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0727fc700 1 -- 172.21.15.85:0/2480364391 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0740137e0 con 0x7fd08415b850 2022-11-27T15:14:33.757 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd08bd64700 1 -- 172.21.15.85:0/2480364391 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0840c4a60 con 0x7fd08415b850 2022-11-27T15:14:33.757 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0727fc700 1 -- 172.21.15.85:0/2480364391 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074013b50 con 0x7fd08415b850 2022-11-27T15:14:33.758 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd08bd64700 1 -- 172.21.15.85:0/2480364391 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0840c5020 con 0x7fd08415b850 2022-11-27T15:14:33.758 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0727fc700 1 -- 172.21.15.85:0/2480364391 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fd074019030 con 0x7fd08415b850 2022-11-27T15:14:33.758 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.743+0000 7fd0727fc700 1 -- 172.21.15.85:0/2480364391 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7fd0740066d0 con 0x7fd08415b850 2022-11-27T15:14:33.758 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.744+0000 7fd08bd64700 1 -- 172.21.15.85:0/2480364391 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0840c5240 con 0x7fd08415b850 2022-11-27T15:14:33.759 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.748+0000 7fd0727fc700 1 -- 172.21.15.85:0/2480364391 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fd074021060 con 0x7fd08415b850 2022-11-27T15:14:33.866 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.864+0000 7fd08bd64700 1 -- 172.21.15.85:0/2480364391 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f6133163-76e1-4748-83ad-e32cfaaaaf41", "id": 5} v 0) v1 -- 0x7fd08404e190 con 0x7fd08415b850 2022-11-27T15:14:33.874 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.872+0000 7fd0727fc700 1 -- 172.21.15.85:0/2480364391 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f6133163-76e1-4748-83ad-e32cfaaaaf41", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd074017070 con 0x7fd08415b850 2022-11-27T15:14:33.874 INFO:teuthology.orchestra.run.smithi085.stdout:5 2022-11-27T15:14:33.876 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.874+0000 7fd08bd64700 1 -- 172.21.15.85:0/2480364391 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415b850 msgr2=0x7fd0840c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:33.877 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.874+0000 7fd08bd64700 1 --2- 172.21.15.85:0/2480364391 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415b850 0x7fd0840c66d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd07400e930 tx=0x7fd07400ec40 comp rx=0 tx=0).stop 2022-11-27T15:14:33.877 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.874+0000 7fd08bd64700 1 -- 172.21.15.85:0/2480364391 shutdown_connections 2022-11-27T15:14:33.878 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.874+0000 7fd08bd64700 1 --2- 172.21.15.85:0/2480364391 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fd08415fdb0 0x7fd084152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.878 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.874+0000 7fd08bd64700 1 --2- 172.21.15.85:0/2480364391 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0840cca40 0x7fd0840c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.878 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.874+0000 7fd08bd64700 1 --2- 172.21.15.85:0/2480364391 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd08415b850 0x7fd0840c66d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:33.879 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.874+0000 7fd08bd64700 1 -- 172.21.15.85:0/2480364391 >> 172.21.15.85:0/2480364391 conn(0x7fd08406de90 msgr2=0x7fd0840cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:33.879 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.874+0000 7fd08bd64700 1 -- 172.21.15.85:0/2480364391 shutdown_connections 2022-11-27T15:14:33.880 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:33.874+0000 7fd08bd64700 1 -- 172.21.15.85:0/2480364391 wait complete. 2022-11-27T15:14:33.887 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 9bdcaeaa-8746-4eb9-bc3c-a3a5f3269550 6 2022-11-27T15:14:33.950 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:33.949+0000 7f6e4d15a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T15:14:34.009 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.007+0000 7facbb8a4700 1 Processor -- start 2022-11-27T15:14:34.009 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.007+0000 7facbb8a4700 1 -- start start 2022-11-27T15:14:34.012 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.008+0000 7facbb8a4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4069860 0x7facb4070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.012 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.008+0000 7facbb8a4700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7facb4070950 0x7facb40cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.012 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.008+0000 7facbb8a4700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb40ca350 con 0x7facb4157c70 2022-11-27T15:14:34.012 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.008+0000 7facbb8a4700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb4070df0 con 0x7facb4070950 2022-11-27T15:14:34.013 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.008+0000 7facbb8a4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb406ad90 con 0x7facb4069860 2022-11-27T15:14:34.013 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.008+0000 7facb9e41700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7facb4070950 0x7facb40cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.013 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.008+0000 7facb8e3f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4069860 0x7facb4070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.013 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.008+0000 7facb9e41700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7facb4070950 0x7facb40cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:39112/0 (socket says 172.21.15.85:39112) 2022-11-27T15:14:34.013 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.008+0000 7facb9e41700 1 -- 172.21.15.85:0/954791439 learned_addr learned my addr 172.21.15.85:0/954791439 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:34.014 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.009+0000 7facb9e41700 1 -- 172.21.15.85:0/954791439 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4069860 msgr2=0x7facb4070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:34.014 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.009+0000 7facb9e41700 1 --2- 172.21.15.85:0/954791439 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4069860 0x7facb4070410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.014 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.009+0000 7facb9e41700 1 -- 172.21.15.85:0/954791439 >> v1:172.21.15.85:6789/0 conn(0x7facb4157c70 legacy=0x7facb40c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.014 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.009+0000 7facb9e41700 1 -- 172.21.15.85:0/954791439 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb4069cf0 con 0x7facb4070950 2022-11-27T15:14:34.015 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.009+0000 7faca3fff700 1 -- 172.21.15.85:0/954791439 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558543357 0 0) 0x7facb40ca350 con 0x7facb4157c70 2022-11-27T15:14:34.015 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.009+0000 7facb8e3f700 1 --2- 172.21.15.85:0/954791439 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4069860 0x7facb4070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:34.015 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.009+0000 7facb9e41700 1 --2- 172.21.15.85:0/954791439 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4070950 0x7facb40cbf60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7facac009e80 tx=0x7facac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2afb7490995652a8 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:34.016 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.009+0000 7faca3fff700 1 -- 172.21.15.85:0/954791439 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac00e070 con 0x7facb4070950 2022-11-27T15:14:34.016 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.009+0000 7faca3fff700 1 -- 172.21.15.85:0/954791439 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac00b8d0 con 0x7facb4070950 2022-11-27T15:14:34.016 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.009+0000 7faca3fff700 1 -- 172.21.15.85:0/954791439 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac00bc80 con 0x7facb4070950 2022-11-27T15:14:34.016 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.010+0000 7facbb8a4700 1 -- 172.21.15.85:0/954791439 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4070950 msgr2=0x7facb40cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.017 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.010+0000 7facbb8a4700 1 --2- 172.21.15.85:0/954791439 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4070950 0x7facb40cbf60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7facac009e80 tx=0x7facac00b330 comp rx=0 tx=0).stop 2022-11-27T15:14:34.017 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.010+0000 7facbb8a4700 1 -- 172.21.15.85:0/954791439 shutdown_connections 2022-11-27T15:14:34.017 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.010+0000 7facbb8a4700 1 --2- 172.21.15.85:0/954791439 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4070950 0x7facb40cbf60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.017 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.010+0000 7facbb8a4700 1 --2- 172.21.15.85:0/954791439 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4069860 0x7facb4070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.017 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.010+0000 7facbb8a4700 1 -- 172.21.15.85:0/954791439 >> 172.21.15.85:0/954791439 conn(0x7facb40aff10 msgr2=0x7facb40b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:34.018 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.010+0000 7facbb8a4700 1 -- 172.21.15.85:0/954791439 shutdown_connections 2022-11-27T15:14:34.018 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.010+0000 7facbb8a4700 1 -- 172.21.15.85:0/954791439 wait complete. 2022-11-27T15:14:34.019 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:34.016+0000 7f2b58316200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T15:14:34.022 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.012+0000 7facbb8a4700 1 Processor -- start 2022-11-27T15:14:34.022 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.012+0000 7facbb8a4700 1 -- start start 2022-11-27T15:14:34.022 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.012+0000 7facbb8a4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4069860 0x7facb40b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.023 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.012+0000 7facbb8a4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4070950 0x7facb415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.023 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.012+0000 7facbb8a4700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7facb4157c70 0x7facb415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.023 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facbb8a4700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb40c2bf0 con 0x7facb4157c70 2022-11-27T15:14:34.023 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facbb8a4700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb40af110 con 0x7facb4069860 2022-11-27T15:14:34.023 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facbb8a4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb40aee30 con 0x7facb4070950 2022-11-27T15:14:34.024 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb9640700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4069860 0x7facb40b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.024 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb9640700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4069860 0x7facb40b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:39114/0 (socket says 172.21.15.85:39114) 2022-11-27T15:14:34.024 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb9640700 1 -- 172.21.15.85:0/2922773531 learned_addr learned my addr 172.21.15.85:0/2922773531 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:34.024 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb8e3f700 1 --2- 172.21.15.85:0/2922773531 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4070950 0x7facb415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.025 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb9e41700 1 --2- 172.21.15.85:0/2922773531 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7facb4157c70 0x7facb415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.025 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb9640700 1 -- 172.21.15.85:0/2922773531 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4070950 msgr2=0x7facb415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.025 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb9640700 1 --2- 172.21.15.85:0/2922773531 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4070950 0x7facb415a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.025 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb9640700 1 -- 172.21.15.85:0/2922773531 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7facb4157c70 msgr2=0x7facb415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.026 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb9640700 1 --2- 172.21.15.85:0/2922773531 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7facb4157c70 0x7facb415a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.026 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb9640700 1 -- 172.21.15.85:0/2922773531 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca400b740 con 0x7facb4069860 2022-11-27T15:14:34.026 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facb9640700 1 --2- 172.21.15.85:0/2922773531 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4069860 0x7facb40b14c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7facb000c550 tx=0x7facb000c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:34.026 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7faca1ffb700 1 -- 172.21.15.85:0/2922773531 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb00054e0 con 0x7facb4069860 2022-11-27T15:14:34.026 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7faca1ffb700 1 -- 172.21.15.85:0/2922773531 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb0003680 con 0x7facb4069860 2022-11-27T15:14:34.027 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7facbb8a4700 1 -- 172.21.15.85:0/2922773531 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac009b30 con 0x7facb4069860 2022-11-27T15:14:34.027 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7faca1ffb700 1 -- 172.21.15.85:0/2922773531 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0003a30 con 0x7facb4069860 2022-11-27T15:14:34.027 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.013+0000 7faca1ffb700 1 -- 172.21.15.85:0/2922773531 <== mon.1 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7facb0019030 con 0x7facb4069860 2022-11-27T15:14:34.027 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.014+0000 7facbb8a4700 1 -- 172.21.15.85:0/2922773531 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb40c2880 con 0x7facb4069860 2022-11-27T15:14:34.028 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.014+0000 7faca1ffb700 1 -- 172.21.15.85:0/2922773531 <== mon.1 v2:172.21.15.85:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7facb0012400 con 0x7facb4069860 2022-11-27T15:14:34.028 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.014+0000 7facbb8a4700 1 -- 172.21.15.85:0/2922773531 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb404e190 con 0x7facb4069860 2022-11-27T15:14:34.028 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.018+0000 7faca1ffb700 1 -- 172.21.15.85:0/2922773531 <== mon.1 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7facb0017070 con 0x7facb4069860 2022-11-27T15:14:34.102 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.101+0000 7facbb8a4700 1 -- 172.21.15.85:0/2922773531 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9bdcaeaa-8746-4eb9-bc3c-a3a5f3269550", "id": 6} v 0) v1 -- 0x7facb4069fd0 con 0x7facb4069860 2022-11-27T15:14:34.112 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:34.109+0000 7f6e4d15a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T15:14:34.113 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.112+0000 7faca1ffb700 1 -- 172.21.15.85:0/2922773531 <== mon.1 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9bdcaeaa-8746-4eb9-bc3c-a3a5f3269550", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7facb0003c60 con 0x7facb4069860 2022-11-27T15:14:34.113 INFO:teuthology.orchestra.run.smithi085.stdout:6 2022-11-27T15:14:34.115 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.113+0000 7facbb8a4700 1 -- 172.21.15.85:0/2922773531 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4069860 msgr2=0x7facb40b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.115 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.113+0000 7facbb8a4700 1 --2- 172.21.15.85:0/2922773531 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4069860 0x7facb40b14c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7facb000c550 tx=0x7facb000c860 comp rx=0 tx=0).stop 2022-11-27T15:14:34.115 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.113+0000 7facbb8a4700 1 -- 172.21.15.85:0/2922773531 shutdown_connections 2022-11-27T15:14:34.115 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.113+0000 7facbb8a4700 1 --2- 172.21.15.85:0/2922773531 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7facb4157c70 0x7facb415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.116 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.113+0000 7facbb8a4700 1 --2- 172.21.15.85:0/2922773531 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7facb4069860 0x7facb40b14c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.116 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.113+0000 7facbb8a4700 1 --2- 172.21.15.85:0/2922773531 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7facb4070950 0x7facb415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.116 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.113+0000 7facbb8a4700 1 -- 172.21.15.85:0/2922773531 >> 172.21.15.85:0/2922773531 conn(0x7facb40aff10 msgr2=0x7facb406e310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:34.116 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.113+0000 7facbb8a4700 1 -- 172.21.15.85:0/2922773531 shutdown_connections 2022-11-27T15:14:34.117 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.113+0000 7facbb8a4700 1 -- 172.21.15.85:0/2922773531 wait complete. 2022-11-27T15:14:34.126 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 0aba1feb-9777-4a35-892d-f1ff9fec1d80 7 2022-11-27T15:14:34.177 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:34.176+0000 7f2b58316200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T15:14:34.247 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.246+0000 7fbc74ad6700 1 Processor -- start 2022-11-27T15:14:34.247 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.246+0000 7fbc74ad6700 1 -- start start 2022-11-27T15:14:34.250 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.247+0000 7fbc74ad6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc70158630 0x7fbc700cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.250 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.247+0000 7fbc74ad6700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fbc700b0dd0 0x7fbc700ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.251 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.247+0000 7fbc74ad6700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc700ce5f0 con 0x7fbc700cd540 2022-11-27T15:14:34.251 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.247+0000 7fbc74ad6700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc700cc810 con 0x7fbc700b0dd0 2022-11-27T15:14:34.251 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.247+0000 7fbc74ad6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700ce310 con 0x7fbc70158630 2022-11-27T15:14:34.251 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.247+0000 7fbc6e59c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc70158630 0x7fbc700cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.252 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.247+0000 7fbc6ed9d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fbc700b0dd0 0x7fbc700ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.252 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.247+0000 7fbc6e59c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc70158630 0x7fbc700cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:37006/0 (socket says 172.21.15.85:37006) 2022-11-27T15:14:34.252 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.247+0000 7fbc6ed9d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fbc700b0dd0 0x7fbc700ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:39124/0 (socket says 172.21.15.85:39124) 2022-11-27T15:14:34.252 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.247+0000 7fbc6e59c700 1 -- 172.21.15.85:0/3580434904 learned_addr learned my addr 172.21.15.85:0/3580434904 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:34.253 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc6e59c700 1 -- 172.21.15.85:0/3580434904 >> v2:172.21.15.85:3300/0 conn(0x7fbc700b0dd0 msgr2=0x7fbc700ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:34.253 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc6e59c700 1 --2- 172.21.15.85:0/3580434904 >> v2:172.21.15.85:3300/0 conn(0x7fbc700b0dd0 0x7fbc700ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.253 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc6e59c700 1 -- 172.21.15.85:0/3580434904 >> v1:172.21.15.35:6789/0 conn(0x7fbc700cd540 legacy=0x7fbc700b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.253 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc6e59c700 1 -- 172.21.15.85:0/3580434904 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc700cca60 con 0x7fbc70158630 2022-11-27T15:14:34.253 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc6ed9d700 1 --2- 172.21.15.85:0/3580434904 >> v2:172.21.15.85:3300/0 conn(0x7fbc700b0dd0 0x7fbc700ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:34.254 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc6e59c700 1 --2- 172.21.15.85:0/3580434904 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc70158630 0x7fbc700cd9e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbc60006fd0 tx=0x7fbc60009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3c6e02e9c6264 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:34.254 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc6d59a700 1 -- 172.21.15.85:0/3580434904 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60005e30 con 0x7fbc70158630 2022-11-27T15:14:34.254 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc6d59a700 1 -- 172.21.15.85:0/3580434904 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc60004030 con 0x7fbc70158630 2022-11-27T15:14:34.254 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc6d59a700 1 -- 172.21.15.85:0/3580434904 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc600043e0 con 0x7fbc70158630 2022-11-27T15:14:34.255 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc74ad6700 1 -- 172.21.15.85:0/3580434904 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc70158630 msgr2=0x7fbc700cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.255 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.248+0000 7fbc74ad6700 1 --2- 172.21.15.85:0/3580434904 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc70158630 0x7fbc700cd9e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbc60006fd0 tx=0x7fbc60009bd0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.255 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.249+0000 7fbc74ad6700 1 -- 172.21.15.85:0/3580434904 shutdown_connections 2022-11-27T15:14:34.255 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.249+0000 7fbc74ad6700 1 --2- 172.21.15.85:0/3580434904 >> v2:172.21.15.85:3300/0 conn(0x7fbc700b0dd0 0x7fbc700ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.255 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.249+0000 7fbc74ad6700 1 --2- 172.21.15.85:0/3580434904 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc70158630 0x7fbc700cd9e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.256 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.249+0000 7fbc74ad6700 1 -- 172.21.15.85:0/3580434904 >> 172.21.15.85:0/3580434904 conn(0x7fbc7006d9c0 msgr2=0x7fbc700cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:34.256 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.249+0000 7fbc74ad6700 1 -- 172.21.15.85:0/3580434904 shutdown_connections 2022-11-27T15:14:34.256 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.249+0000 7fbc74ad6700 1 -- 172.21.15.85:0/3580434904 wait complete. 2022-11-27T15:14:34.256 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.250+0000 7fbc74ad6700 1 Processor -- start 2022-11-27T15:14:34.257 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.250+0000 7fbc74ad6700 1 -- start start 2022-11-27T15:14:34.259 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc74ad6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc700b0dd0 0x7fbc700c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.259 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc74ad6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbc700cd540 0x7fbc70154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.260 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc74ad6700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbc70158630 0x7fbc7014cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.260 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc74ad6700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70157c00 con 0x7fbc70158630 2022-11-27T15:14:34.260 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc74ad6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700c7d10 con 0x7fbc700cd540 2022-11-27T15:14:34.260 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc74ad6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7014d1b0 con 0x7fbc700b0dd0 2022-11-27T15:14:34.261 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6e59c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc700b0dd0 0x7fbc700c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.261 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6dd9b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbc700cd540 0x7fbc70154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.261 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6e59c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc700b0dd0 0x7fbc700c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:37012/0 (socket says 172.21.15.85:37012) 2022-11-27T15:14:34.261 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6e59c700 1 -- 172.21.15.85:0/4179058175 learned_addr learned my addr 172.21.15.85:0/4179058175 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-11-27T15:14:34.262 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6ed9d700 1 --2- 172.21.15.85:0/4179058175 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbc70158630 0x7fbc7014cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.262 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6e59c700 1 -- 172.21.15.85:0/4179058175 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbc700cd540 msgr2=0x7fbc70154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.262 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6e59c700 1 --2- 172.21.15.85:0/4179058175 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbc700cd540 0x7fbc70154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.262 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6e59c700 1 -- 172.21.15.85:0/4179058175 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbc70158630 msgr2=0x7fbc7014cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.262 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6e59c700 1 --2- 172.21.15.85:0/4179058175 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbc70158630 0x7fbc7014cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.263 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6e59c700 1 -- 172.21.15.85:0/4179058175 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc68009a70 con 0x7fbc700b0dd0 2022-11-27T15:14:34.263 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6dd9b700 1 --2- 172.21.15.85:0/4179058175 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbc700cd540 0x7fbc70154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:34.263 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc6e59c700 1 --2- 172.21.15.85:0/4179058175 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc700b0dd0 0x7fbc700c6b20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc60000c00 tx=0x7fbc60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:34.263 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc5effd700 1 -- 172.21.15.85:0/4179058175 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60003ce0 con 0x7fbc700b0dd0 2022-11-27T15:14:34.264 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc5effd700 1 -- 172.21.15.85:0/4179058175 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc60003e60 con 0x7fbc700b0dd0 2022-11-27T15:14:34.264 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc74ad6700 1 -- 172.21.15.85:0/4179058175 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc60009500 con 0x7fbc700b0dd0 2022-11-27T15:14:34.264 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.251+0000 7fbc5effd700 1 -- 172.21.15.85:0/4179058175 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6001c660 con 0x7fbc700b0dd0 2022-11-27T15:14:34.264 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.252+0000 7fbc74ad6700 1 -- 172.21.15.85:0/4179058175 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc700c73e0 con 0x7fbc700b0dd0 2022-11-27T15:14:34.264 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.252+0000 7fbc5effd700 1 -- 172.21.15.85:0/4179058175 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fbc60022030 con 0x7fbc700b0dd0 2022-11-27T15:14:34.265 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.252+0000 7fbc5effd700 1 -- 172.21.15.85:0/4179058175 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fbc60011a80 con 0x7fbc700b0dd0 2022-11-27T15:14:34.265 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.252+0000 7fbc74ad6700 1 -- 172.21.15.85:0/4179058175 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7004e190 con 0x7fbc700b0dd0 2022-11-27T15:14:34.265 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.254+0000 7fbc5effd700 1 -- 172.21.15.85:0/4179058175 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fbc600446d0 con 0x7fbc700b0dd0 2022-11-27T15:14:34.338 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.338+0000 7fbc74ad6700 1 -- 172.21.15.85:0/4179058175 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0aba1feb-9777-4a35-892d-f1ff9fec1d80", "id": 7} v 0) v1 -- 0x7fbc700ccd40 con 0x7fbc700b0dd0 2022-11-27T15:14:34.346 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.346+0000 7fbc5effd700 1 -- 172.21.15.85:0/4179058175 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0aba1feb-9777-4a35-892d-f1ff9fec1d80", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbc60033020 con 0x7fbc700b0dd0 2022-11-27T15:14:34.346 INFO:teuthology.orchestra.run.smithi085.stdout:7 2022-11-27T15:14:34.347 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.346+0000 7fbc74ad6700 1 -- 172.21.15.85:0/4179058175 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc700b0dd0 msgr2=0x7fbc700c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.348 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.346+0000 7fbc74ad6700 1 --2- 172.21.15.85:0/4179058175 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc700b0dd0 0x7fbc700c6b20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc60000c00 tx=0x7fbc60004750 comp rx=0 tx=0).stop 2022-11-27T15:14:34.348 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.346+0000 7fbc74ad6700 1 -- 172.21.15.85:0/4179058175 shutdown_connections 2022-11-27T15:14:34.348 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.346+0000 7fbc74ad6700 1 --2- 172.21.15.85:0/4179058175 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbc70158630 0x7fbc7014cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.348 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.346+0000 7fbc74ad6700 1 --2- 172.21.15.85:0/4179058175 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbc700b0dd0 0x7fbc700c6b20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.349 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.346+0000 7fbc74ad6700 1 --2- 172.21.15.85:0/4179058175 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbc700cd540 0x7fbc70154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.349 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.346+0000 7fbc74ad6700 1 -- 172.21.15.85:0/4179058175 >> 172.21.15.85:0/4179058175 conn(0x7fbc7006d9c0 msgr2=0x7fbc700cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:34.349 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.347+0000 7fbc74ad6700 1 -- 172.21.15.85:0/4179058175 shutdown_connections 2022-11-27T15:14:34.349 INFO:teuthology.orchestra.run.smithi085.stderr:2022-11-27T15:14:34.347+0000 7fbc74ad6700 1 -- 172.21.15.85:0/4179058175 wait complete. 2022-11-27T15:14:34.358 INFO:tasks.ceph.osd.0:Restarting daemon 2022-11-27T15:14:34.359 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-11-27T15:14:34.361 INFO:tasks.ceph.osd.0:Started 2022-11-27T15:14:34.362 INFO:tasks.ceph.osd.1:Restarting daemon 2022-11-27T15:14:34.362 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-11-27T15:14:34.364 INFO:tasks.ceph.osd.1:Started 2022-11-27T15:14:34.364 INFO:tasks.ceph.osd.2:Restarting daemon 2022-11-27T15:14:34.365 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-11-27T15:14:34.367 INFO:tasks.ceph.osd.2:Started 2022-11-27T15:14:34.368 INFO:tasks.ceph.osd.3:Restarting daemon 2022-11-27T15:14:34.368 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-11-27T15:14:34.370 INFO:tasks.ceph.osd.3:Started 2022-11-27T15:14:34.371 INFO:tasks.ceph.osd.4:Restarting daemon 2022-11-27T15:14:34.371 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-11-27T15:14:34.373 INFO:tasks.ceph.osd.4:Started 2022-11-27T15:14:34.374 INFO:tasks.ceph.osd.5:Restarting daemon 2022-11-27T15:14:34.374 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-11-27T15:14:34.376 INFO:tasks.ceph.osd.5:Started 2022-11-27T15:14:34.376 INFO:tasks.ceph.osd.6:Restarting daemon 2022-11-27T15:14:34.377 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-11-27T15:14:34.379 INFO:tasks.ceph.osd.6:Started 2022-11-27T15:14:34.379 INFO:tasks.ceph.osd.7:Restarting daemon 2022-11-27T15:14:34.380 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-11-27T15:14:34.382 INFO:tasks.ceph.osd.7:Started 2022-11-27T15:14:34.382 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T15:14:34.546 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.545+0000 7f917af5c700 1 Processor -- start 2022-11-27T15:14:34.546 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.545+0000 7f917af5c700 1 -- start start 2022-11-27T15:14:34.550 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.546+0000 7f917af5c700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f91740b17b0 0x7f917415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.550 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.546+0000 7f917af5c700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f91740cdfc0 0x7f91740ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.551 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.546+0000 7f917af5c700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9174159b10 con 0x7f91740cbff0 2022-11-27T15:14:34.551 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.546+0000 7f917af5c700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91740cdba0 con 0x7f91740cdfc0 2022-11-27T15:14:34.551 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.546+0000 7f917af5c700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91740ca380 con 0x7f91740b17b0 2022-11-27T15:14:34.551 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.546+0000 7f91794f9700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f91740cdfc0 0x7f91740ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.552 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91794f9700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f91740cdfc0 0x7f91740ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39418/0 (socket says 172.21.15.35:39418) 2022-11-27T15:14:34.552 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91794f9700 1 -- 172.21.15.35:0/4157190409 learned_addr learned my addr 172.21.15.35:0/4157190409 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:34.553 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f9173fff700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f91740cbff0 0x7f91740b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50812/0 (socket says 172.21.15.35:50812) 2022-11-27T15:14:34.553 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f9178cf8700 1 --2- 172.21.15.35:0/4157190409 >> v2:172.21.15.85:3300/0 conn(0x7f91740b17b0 0x7f917415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.553 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91794f9700 1 -- 172.21.15.35:0/4157190409 >> v2:172.21.15.85:3300/0 conn(0x7f91740b17b0 msgr2=0x7f917415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:34.553 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91794f9700 1 --2- 172.21.15.35:0/4157190409 >> v2:172.21.15.85:3300/0 conn(0x7f91740b17b0 0x7f917415dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.553 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91794f9700 1 -- 172.21.15.35:0/4157190409 >> v1:172.21.15.35:6789/0 conn(0x7f91740cbff0 legacy=0x7f91740b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.554 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91794f9700 1 -- 172.21.15.35:0/4157190409 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91740af530 con 0x7f91740cdfc0 2022-11-27T15:14:34.554 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91737fe700 1 -- 172.21.15.35:0/4157190409 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90276008 0 0) 0x7f9160001110 con 0x7f91740cbff0 2022-11-27T15:14:34.554 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91794f9700 1 --2- 172.21.15.35:0/4157190409 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 0x7f91740ce3d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f91740715d0 tx=0x7f916c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4799ffe4809ef52 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:34.554 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91737fe700 1 -- 172.21.15.35:0/4157190409 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c00e070 con 0x7f91740cdfc0 2022-11-27T15:14:34.555 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91737fe700 1 -- 172.21.15.35:0/4157190409 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c00b910 con 0x7f91740cdfc0 2022-11-27T15:14:34.555 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.547+0000 7f91737fe700 1 -- 172.21.15.35:0/4157190409 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c00bcc0 con 0x7f91740cdfc0 2022-11-27T15:14:34.555 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.548+0000 7f917af5c700 1 -- 172.21.15.35:0/4157190409 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 msgr2=0x7f91740ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.555 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.548+0000 7f917af5c700 1 --2- 172.21.15.35:0/4157190409 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 0x7f91740ce3d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f91740715d0 tx=0x7f916c00b330 comp rx=0 tx=0).stop 2022-11-27T15:14:34.556 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.548+0000 7f917af5c700 1 -- 172.21.15.35:0/4157190409 shutdown_connections 2022-11-27T15:14:34.556 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.548+0000 7f917af5c700 1 --2- 172.21.15.35:0/4157190409 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 0x7f91740ce3d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.556 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.548+0000 7f917af5c700 1 --2- 172.21.15.35:0/4157190409 >> v2:172.21.15.85:3300/0 conn(0x7f91740b17b0 0x7f917415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.556 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.548+0000 7f917af5c700 1 -- 172.21.15.35:0/4157190409 >> 172.21.15.35:0/4157190409 conn(0x7f917406eb60 msgr2=0x7f91740cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:34.557 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.548+0000 7f917af5c700 1 -- 172.21.15.35:0/4157190409 shutdown_connections 2022-11-27T15:14:34.557 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.548+0000 7f917af5c700 1 -- 172.21.15.35:0/4157190409 wait complete. 2022-11-27T15:14:34.557 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.549+0000 7f917af5c700 1 Processor -- start 2022-11-27T15:14:34.557 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.549+0000 7f917af5c700 1 -- start start 2022-11-27T15:14:34.561 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f917af5c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91740b17b0 0x7f91741500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.561 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f917af5c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91740cbff0 0x7f917406bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.562 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f917af5c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 0x7f91740c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:34.562 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f917af5c700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917415cbc0 con 0x7f91740cdfc0 2022-11-27T15:14:34.562 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f917af5c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917415b710 con 0x7f91740b17b0 2022-11-27T15:14:34.562 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f917af5c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917415c200 con 0x7f91740cbff0 2022-11-27T15:14:34.562 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f91794f9700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 0x7f91740c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.563 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f91794f9700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 0x7f91740c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39424/0 (socket says 172.21.15.35:39424) 2022-11-27T15:14:34.563 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f91794f9700 1 -- 172.21.15.35:0/754374703 learned_addr learned my addr 172.21.15.35:0/754374703 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:34.563 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f9178cf8700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91740b17b0 0x7f91741500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.564 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f9173fff700 1 --2- 172.21.15.35:0/754374703 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91740cbff0 0x7f917406bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:34.564 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f91794f9700 1 -- 172.21.15.35:0/754374703 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91740cbff0 msgr2=0x7f917406bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.564 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f91794f9700 1 --2- 172.21.15.35:0/754374703 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91740cbff0 0x7f917406bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.564 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f91794f9700 1 -- 172.21.15.35:0/754374703 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91740b17b0 msgr2=0x7f91741500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.565 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f91794f9700 1 --2- 172.21.15.35:0/754374703 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91740b17b0 0x7f91741500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.565 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.550+0000 7f91794f9700 1 -- 172.21.15.35:0/754374703 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c009b30 con 0x7f91740cdfc0 2022-11-27T15:14:34.565 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.551+0000 7f91794f9700 1 --2- 172.21.15.35:0/754374703 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 0x7f91740c8e10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f916c00b680 tx=0x7f916c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:34.565 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.551+0000 7f91717fa700 1 -- 172.21.15.35:0/754374703 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0048b0 con 0x7f91740cdfc0 2022-11-27T15:14:34.565 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.551+0000 7f91717fa700 1 -- 172.21.15.35:0/754374703 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c0099b0 con 0x7f91740cdfc0 2022-11-27T15:14:34.566 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.551+0000 7f917af5c700 1 -- 172.21.15.35:0/754374703 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917415b960 con 0x7f91740cdfc0 2022-11-27T15:14:34.566 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.551+0000 7f91717fa700 1 -- 172.21.15.35:0/754374703 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c01c680 con 0x7f91740cdfc0 2022-11-27T15:14:34.566 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.551+0000 7f917af5c700 1 -- 172.21.15.35:0/754374703 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917415bf20 con 0x7f91740cdfc0 2022-11-27T15:14:34.567 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.551+0000 7f91717fa700 1 -- 172.21.15.35:0/754374703 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f916c022030 con 0x7f91740cdfc0 2022-11-27T15:14:34.567 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.551+0000 7f91717fa700 1 -- 172.21.15.35:0/754374703 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f916c012ac0 con 0x7f91740cdfc0 2022-11-27T15:14:34.567 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.552+0000 7f917af5c700 1 -- 172.21.15.35:0/754374703 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917404e190 con 0x7f91740cdfc0 2022-11-27T15:14:34.567 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.554+0000 7f91717fa700 1 -- 172.21.15.35:0/754374703 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f916c020070 con 0x7f91740cdfc0 2022-11-27T15:14:34.626 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.624+0000 7f917af5c700 1 -- 172.21.15.35:0/754374703 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f917415c420 con 0x7f91740cdfc0 2022-11-27T15:14:34.627 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.625+0000 7f91717fa700 1 -- 172.21.15.35:0/754374703 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f916c012d20 con 0x7f91740cdfc0 2022-11-27T15:14:34.627 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:14:34.628 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":10,"fsid":"1a1c2d0e-bda9-4965-9029-a4d82eabd38a","created":"2022-11-27T15:14:31.789352+0000","modified":"2022-11-27T15:14:34.339079+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T15:14:34.339079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"92e0fd37-a176-44da-847e-2bc153faa513","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e9828c4c-3cab-4b51-80fe-beaca70fd85a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"80ba487d-307a-46d3-bf27-fb037321414c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"962bb8b8-e6be-48ce-99c8-4697ba5fd364","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fd1b4ee8-0942-467c-b3cc-990bfb7c34c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f6133163-76e1-4748-83ad-e32cfaaaaf41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9bdcaeaa-8746-4eb9-bc3c-a3a5f3269550","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0aba1feb-9777-4a35-892d-f1ff9fec1d80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T15:14:34.630 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.626+0000 7f917af5c700 1 -- 172.21.15.35:0/754374703 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 msgr2=0x7f91740c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:34.630 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.626+0000 7f917af5c700 1 --2- 172.21.15.35:0/754374703 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 0x7f91740c8e10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f916c00b680 tx=0x7f916c004ad0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.631 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.626+0000 7f917af5c700 1 -- 172.21.15.35:0/754374703 shutdown_connections 2022-11-27T15:14:34.631 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.626+0000 7f917af5c700 1 --2- 172.21.15.35:0/754374703 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f91740cdfc0 0x7f91740c8e10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.632 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.626+0000 7f917af5c700 1 --2- 172.21.15.35:0/754374703 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91740b17b0 0x7f91741500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.632 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.626+0000 7f917af5c700 1 --2- 172.21.15.35:0/754374703 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91740cbff0 0x7f917406bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:34.632 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.626+0000 7f917af5c700 1 -- 172.21.15.35:0/754374703 >> 172.21.15.35:0/754374703 conn(0x7f917406eb60 msgr2=0x7f9174158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:34.633 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.626+0000 7f917af5c700 1 -- 172.21.15.35:0/754374703 shutdown_connections 2022-11-27T15:14:34.634 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:34.626+0000 7f917af5c700 1 -- 172.21.15.35:0/754374703 wait complete. 2022-11-27T15:14:34.640 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-11-27T15:14:34.641 INFO:tasks.ceph:Waiting for OSDs to come up 2022-11-27T15:14:34.724 INFO:tasks.ceph.osd.4.smithi085.stderr:2022-11-27T15:14:34.722+0000 7fd18e9a35c0 -1 Falling back to public interface 2022-11-27T15:14:34.735 INFO:tasks.ceph.osd.0.smithi035.stderr:2022-11-27T15:14:34.734+0000 7f22c69395c0 -1 Falling back to public interface 2022-11-27T15:14:34.837 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:34.836+0000 7f6e4d15a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T15:14:34.912 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:34.911+0000 7f6e4d15a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T15:14:34.929 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:34.928+0000 7f2b58316200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T15:14:34.978 INFO:tasks.ceph.osd.5.smithi085.stderr:2022-11-27T15:14:34.976+0000 7f6dc228a5c0 -1 Falling back to public interface 2022-11-27T15:14:34.985 INFO:tasks.ceph.osd.6.smithi085.stderr:2022-11-27T15:14:34.984+0000 7f471f63a5c0 -1 Falling back to public interface 2022-11-27T15:14:34.989 INFO:tasks.ceph.osd.2.smithi035.stderr:2022-11-27T15:14:34.988+0000 7f2d7a5f85c0 -1 Falling back to public interface 2022-11-27T15:14:35.002 INFO:tasks.ceph.osd.1.smithi035.stderr:2022-11-27T15:14:35.001+0000 7f6bdfb3d5c0 -1 Falling back to public interface 2022-11-27T15:14:35.008 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:35.008+0000 7f2b58316200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T15:14:35.009 INFO:tasks.ceph.osd.3.smithi035.stderr:2022-11-27T15:14:35.008+0000 7f3249d985c0 -1 Falling back to public interface 2022-11-27T15:14:35.018 INFO:tasks.ceph.osd.7.smithi085.stderr:2022-11-27T15:14:35.017+0000 7f3faedf55c0 -1 Falling back to public interface 2022-11-27T15:14:35.051 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:35.049+0000 7f6e4d15a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T15:14:35.152 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:35.151+0000 7f2b58316200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T15:14:35.206 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:35.205+0000 7f6e4d15a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T15:14:35.317 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:35.316+0000 7f2b58316200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T15:14:35.338 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:35.336+0000 7f6e4d15a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T15:14:35.447 DEBUG:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T15:14:35.454 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:35.453+0000 7f2b58316200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T15:14:35.565 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.564+0000 7f242cd14700 1 Processor -- start 2022-11-27T15:14:35.566 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.564+0000 7f242cd14700 1 -- start start 2022-11-27T15:14:35.569 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.565+0000 7f242cd14700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f242815cad0 0x7f24280ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:35.569 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.565+0000 7f242cd14700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f24280b2c50 0x7f24280b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:35.569 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.565+0000 7f242cd14700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280b26e0 con 0x7f24280ce670 2022-11-27T15:14:35.569 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.565+0000 7f242cd14700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242815cea0 con 0x7f242815cad0 2022-11-27T15:14:35.570 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.565+0000 7f242cd14700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24280ca830 con 0x7f24280b2c50 2022-11-27T15:14:35.570 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f2426d9d700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f24280ce670 0x7f24280b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50824/0 (socket says 172.21.15.35:50824) 2022-11-27T15:14:35.570 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f2426d9d700 1 -- 172.21.15.35:0/2724193016 learned_addr learned my addr 172.21.15.35:0/2724193016 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-27T15:14:35.570 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242659c700 1 --2- 172.21.15.35:0/2724193016 >> v2:172.21.15.85:3300/0 conn(0x7f242815cad0 0x7f24280ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:35.571 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f2425d9b700 1 --2- 172.21.15.35:0/2724193016 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f24280b2c50 0x7f24280b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:35.571 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242559a700 1 -- 172.21.15.35:0/2724193016 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253712918 0 0) 0x7f24280b26e0 con 0x7f24280ce670 2022-11-27T15:14:35.571 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242559a700 1 -- 172.21.15.35:0/2724193016 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f240c0035b0 con 0x7f24280ce670 2022-11-27T15:14:35.571 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242659c700 1 -- 172.21.15.35:0/2724193016 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f24280b2c50 msgr2=0x7f24280b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:35.572 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242659c700 1 --2- 172.21.15.35:0/2724193016 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f24280b2c50 0x7f24280b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:35.572 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242659c700 1 -- 172.21.15.35:0/2724193016 >> v1:172.21.15.35:6789/0 conn(0x7f24280ce670 legacy=0x7f24280b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:35.572 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242659c700 1 -- 172.21.15.35:0/2724193016 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280cdb90 con 0x7f242815cad0 2022-11-27T15:14:35.572 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242659c700 1 --2- 172.21.15.35:0/2724193016 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f242815cad0 0x7f24280ccbb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2418009840 tx=0x7f2418009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eaa558006fdd9c1 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:35.573 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242559a700 1 -- 172.21.15.35:0/2724193016 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418011830 con 0x7f242815cad0 2022-11-27T15:14:35.573 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242559a700 1 -- 172.21.15.35:0/2724193016 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418011990 con 0x7f242815cad0 2022-11-27T15:14:35.573 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.566+0000 7f242559a700 1 -- 172.21.15.35:0/2724193016 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418011d20 con 0x7f242815cad0 2022-11-27T15:14:35.573 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.567+0000 7f242cd14700 1 -- 172.21.15.35:0/2724193016 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f242815cad0 msgr2=0x7f24280ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:35.574 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.567+0000 7f242cd14700 1 --2- 172.21.15.35:0/2724193016 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f242815cad0 0x7f24280ccbb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2418009840 tx=0x7f2418009b50 comp rx=0 tx=0).stop 2022-11-27T15:14:35.574 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.567+0000 7f242cd14700 1 -- 172.21.15.35:0/2724193016 shutdown_connections 2022-11-27T15:14:35.574 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.567+0000 7f242cd14700 1 --2- 172.21.15.35:0/2724193016 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f242815cad0 0x7f24280ccbb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:35.574 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.567+0000 7f242cd14700 1 --2- 172.21.15.35:0/2724193016 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f24280b2c50 0x7f24280b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:35.574 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.567+0000 7f242cd14700 1 -- 172.21.15.35:0/2724193016 >> 172.21.15.35:0/2724193016 conn(0x7f242806df40 msgr2=0x7f242806e310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:35.575 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.567+0000 7f242cd14700 1 -- 172.21.15.35:0/2724193016 shutdown_connections 2022-11-27T15:14:35.575 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.567+0000 7f242cd14700 1 -- 172.21.15.35:0/2724193016 wait complete. 2022-11-27T15:14:35.578 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.569+0000 7f242cd14700 1 Processor -- start 2022-11-27T15:14:35.578 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.569+0000 7f242cd14700 1 -- start start 2022-11-27T15:14:35.578 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.569+0000 7f242cd14700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f24280b2c50 0x7f242815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:35.578 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.569+0000 7f242cd14700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f24280ce670 0x7f24280a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:35.579 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.569+0000 7f242cd14700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f242815cad0 0x7f24281f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:35.579 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.569+0000 7f242cd14700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280c5850 con 0x7f24280ce670 2022-11-27T15:14:35.579 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.569+0000 7f242cd14700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280c5b30 con 0x7f242815cad0 2022-11-27T15:14:35.579 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.569+0000 7f242cd14700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24280c5e10 con 0x7f24280b2c50 2022-11-27T15:14:35.580 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f24280ce670 0x7f24280a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:35.580 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f24280ce670 0x7f24280a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39432/0 (socket says 172.21.15.35:39432) 2022-11-27T15:14:35.580 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2425d9b700 1 -- 172.21.15.35:0/567968124 learned_addr learned my addr 172.21.15.35:0/567968124 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:35.580 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2426d9d700 1 --2- 172.21.15.35:0/567968124 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f242815cad0 0x7f24281f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:35.581 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f242659c700 1 --2- 172.21.15.35:0/567968124 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f24280b2c50 0x7f242815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:35.581 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2425d9b700 1 -- 172.21.15.35:0/567968124 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f24280b2c50 msgr2=0x7f242815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:35.581 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2425d9b700 1 --2- 172.21.15.35:0/567968124 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f24280b2c50 0x7f242815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:35.581 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2425d9b700 1 -- 172.21.15.35:0/567968124 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f242815cad0 msgr2=0x7f24281f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:35.581 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2425d9b700 1 --2- 172.21.15.35:0/567968124 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f242815cad0 0x7f24281f1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:35.582 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2425d9b700 1 -- 172.21.15.35:0/567968124 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24180094a0 con 0x7f24280ce670 2022-11-27T15:14:35.582 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2426d9d700 1 --2- 172.21.15.35:0/567968124 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f242815cad0 0x7f24281f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:35.582 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2425d9b700 1 --2- 172.21.15.35:0/567968124 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f24280ce670 0x7f24280a67b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f241c00e9b0 tx=0x7f241c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:35.582 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2416ffd700 1 -- 172.21.15.35:0/567968124 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c009710 con 0x7f24280ce670 2022-11-27T15:14:35.583 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2416ffd700 1 -- 172.21.15.35:0/567968124 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c00ee50 con 0x7f24280ce670 2022-11-27T15:14:35.583 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f242cd14700 1 -- 172.21.15.35:0/567968124 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24280c60f0 con 0x7f24280ce670 2022-11-27T15:14:35.583 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f2416ffd700 1 -- 172.21.15.35:0/567968124 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00ca50 con 0x7f24280ce670 2022-11-27T15:14:35.583 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.570+0000 7f242cd14700 1 -- 172.21.15.35:0/567968124 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24280a6cf0 con 0x7f24280ce670 2022-11-27T15:14:35.583 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.571+0000 7f2416ffd700 1 -- 172.21.15.35:0/567968124 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f241c019030 con 0x7f24280ce670 2022-11-27T15:14:35.584 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.571+0000 7f2416ffd700 1 -- 172.21.15.35:0/567968124 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f241c006880 con 0x7f24280ce670 2022-11-27T15:14:35.584 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.571+0000 7f242cd14700 1 -- 172.21.15.35:0/567968124 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24280c5c70 con 0x7f24280ce670 2022-11-27T15:14:35.584 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.574+0000 7f2416ffd700 1 -- 172.21.15.35:0/567968124 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f24280c5c70 con 0x7f24280ce670 2022-11-27T15:14:35.645 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.644+0000 7f242cd14700 1 -- 172.21.15.35:0/567968124 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f242804e190 con 0x7f24280ce670 2022-11-27T15:14:35.646 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.645+0000 7f2416ffd700 1 -- 172.21.15.35:0/567968124 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f241c021060 con 0x7f24280ce670 2022-11-27T15:14:35.646 INFO:teuthology.misc.health.smithi035.stdout: 2022-11-27T15:14:35.647 INFO:teuthology.misc.health.smithi035.stdout:{"epoch":10,"fsid":"1a1c2d0e-bda9-4965-9029-a4d82eabd38a","created":"2022-11-27T15:14:31.789352+0000","modified":"2022-11-27T15:14:34.339079+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T15:14:34.339079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"92e0fd37-a176-44da-847e-2bc153faa513","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e9828c4c-3cab-4b51-80fe-beaca70fd85a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"80ba487d-307a-46d3-bf27-fb037321414c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"962bb8b8-e6be-48ce-99c8-4697ba5fd364","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fd1b4ee8-0942-467c-b3cc-990bfb7c34c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f6133163-76e1-4748-83ad-e32cfaaaaf41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9bdcaeaa-8746-4eb9-bc3c-a3a5f3269550","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0aba1feb-9777-4a35-892d-f1ff9fec1d80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T15:14:35.648 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.646+0000 7f242cd14700 1 -- 172.21.15.35:0/567968124 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f24280ce670 msgr2=0x7f24280a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:35.648 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.646+0000 7f242cd14700 1 --2- 172.21.15.35:0/567968124 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f24280ce670 0x7f24280a67b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f241c00e9b0 tx=0x7f241c00ecc0 comp rx=0 tx=0).stop 2022-11-27T15:14:35.648 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.646+0000 7f242cd14700 1 -- 172.21.15.35:0/567968124 shutdown_connections 2022-11-27T15:14:35.649 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.646+0000 7f242cd14700 1 --2- 172.21.15.35:0/567968124 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f242815cad0 0x7f24281f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:35.649 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.646+0000 7f242cd14700 1 --2- 172.21.15.35:0/567968124 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f24280b2c50 0x7f242815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:35.649 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.646+0000 7f242cd14700 1 --2- 172.21.15.35:0/567968124 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f24280ce670 0x7f24280a67b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:35.649 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.646+0000 7f242cd14700 1 -- 172.21.15.35:0/567968124 >> 172.21.15.35:0/567968124 conn(0x7f242806df40 msgr2=0x7f24280c8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:35.649 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.646+0000 7f242cd14700 1 -- 172.21.15.35:0/567968124 shutdown_connections 2022-11-27T15:14:35.650 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:35.646+0000 7f242cd14700 1 -- 172.21.15.35:0/567968124 wait complete. 2022-11-27T15:14:35.658 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-11-27T15:14:35.685 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:35.684+0000 7f6e4d15a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T15:14:35.821 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:35.820+0000 7f2b58316200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T15:14:35.837 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:35.836+0000 7f6e4d15a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T15:14:35.953 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:35.951+0000 7f6e4d15a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T15:14:35.979 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:35.978+0000 7f2b58316200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T15:14:36.095 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:36.094+0000 7f6e4d15a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T15:14:36.100 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:36.099+0000 7f2b58316200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T15:14:36.194 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:36.193+0000 7f6e4d15a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T15:14:36.250 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:36.250+0000 7f2b58316200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T15:14:36.354 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:36.353+0000 7f2b58316200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T15:14:36.546 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:36.545+0000 7f6e4d15a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T15:14:36.727 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:36.726+0000 7f2b58316200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T15:14:37.123 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:37.122+0000 7f6e4d15a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T15:14:37.343 INFO:tasks.ceph.mgr.y.smithi035.stderr:2022-11-27T15:14:37.343+0000 7f2b58316200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T15:14:38.148 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:38.146+0000 7f6e05524700 -1 mgr.server handle_report got status from non-daemon mon.b 2022-11-27T15:14:38.148 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-11-27T15:14:38.146+0000 7f6e05524700 -1 mgr.server handle_report got status from non-daemon mon.c 2022-11-27T15:14:38.244 INFO:tasks.ceph.osd.4.smithi085.stderr:2022-11-27T15:14:38.243+0000 7fd18e9a35c0 -1 osd.4 0 log_to_monitors true 2022-11-27T15:14:38.247 INFO:tasks.ceph.osd.0.smithi035.stderr:2022-11-27T15:14:38.247+0000 7f22c69395c0 -1 osd.0 0 log_to_monitors true 2022-11-27T15:14:38.492 INFO:tasks.ceph.osd.1.smithi035.stderr:2022-11-27T15:14:38.491+0000 7f6bdfb3d5c0 -1 osd.1 0 log_to_monitors true 2022-11-27T15:14:38.505 INFO:tasks.ceph.osd.2.smithi035.stderr:2022-11-27T15:14:38.505+0000 7f2d7a5f85c0 -1 osd.2 0 log_to_monitors true 2022-11-27T15:14:38.515 INFO:tasks.ceph.osd.5.smithi085.stderr:2022-11-27T15:14:38.514+0000 7f6dc228a5c0 -1 osd.5 0 log_to_monitors true 2022-11-27T15:14:38.517 INFO:tasks.ceph.osd.6.smithi085.stderr:2022-11-27T15:14:38.516+0000 7f471f63a5c0 -1 osd.6 0 log_to_monitors true 2022-11-27T15:14:38.539 INFO:tasks.ceph.osd.3.smithi035.stderr:2022-11-27T15:14:38.539+0000 7f3249d985c0 -1 osd.3 0 log_to_monitors true 2022-11-27T15:14:38.586 INFO:tasks.ceph.osd.7.smithi085.stderr:2022-11-27T15:14:38.585+0000 7f3faedf55c0 -1 osd.7 0 log_to_monitors true 2022-11-27T15:14:42.020 INFO:tasks.ceph.osd.2.smithi035.stderr:2022-11-27T15:14:42.018+0000 7f2d6e1af700 -1 osd.2 0 waiting for initial osdmap 2022-11-27T15:14:42.030 INFO:tasks.ceph.osd.1.smithi035.stderr:2022-11-27T15:14:42.029+0000 7f6bd36f4700 -1 osd.1 0 waiting for initial osdmap 2022-11-27T15:14:42.039 INFO:tasks.ceph.osd.2.smithi035.stderr:2022-11-27T15:14:42.039+0000 7f2d6658a700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:14:42.048 INFO:tasks.ceph.osd.1.smithi035.stderr:2022-11-27T15:14:42.048+0000 7f6bcbacf700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:14:42.074 INFO:tasks.ceph.osd.0.smithi035.stderr:2022-11-27T15:14:42.073+0000 7f22ba4f0700 -1 osd.0 0 waiting for initial osdmap 2022-11-27T15:14:42.075 INFO:tasks.ceph.osd.3.smithi035.stderr:2022-11-27T15:14:42.074+0000 7f323d94f700 -1 osd.3 0 waiting for initial osdmap 2022-11-27T15:14:42.091 INFO:tasks.ceph.osd.0.smithi035.stderr:2022-11-27T15:14:42.090+0000 7f22b28cb700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:14:42.092 INFO:tasks.ceph.osd.3.smithi035.stderr:2022-11-27T15:14:42.090+0000 7f3235d2a700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:14:42.097 INFO:tasks.ceph.osd.4.smithi085.stderr:2022-11-27T15:14:42.093+0000 7fd18255a700 -1 osd.4 0 waiting for initial osdmap 2022-11-27T15:14:42.108 INFO:tasks.ceph.osd.4.smithi085.stderr:2022-11-27T15:14:42.106+0000 7fd17a935700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:14:42.161 INFO:tasks.ceph.osd.7.smithi085.stderr:2022-11-27T15:14:42.160+0000 7f3fa29ac700 -1 osd.7 0 waiting for initial osdmap 2022-11-27T15:14:42.179 INFO:tasks.ceph.osd.7.smithi085.stderr:2022-11-27T15:14:42.178+0000 7f3f9ad87700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:14:42.184 INFO:tasks.ceph.osd.5.smithi085.stderr:2022-11-27T15:14:42.183+0000 7f6db5e41700 -1 osd.5 0 waiting for initial osdmap 2022-11-27T15:14:42.196 INFO:tasks.ceph.osd.5.smithi085.stderr:2022-11-27T15:14:42.195+0000 7f6dae21c700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:14:42.227 INFO:tasks.ceph.osd.6.smithi085.stderr:2022-11-27T15:14:42.225+0000 7f47131f1700 -1 osd.6 0 waiting for initial osdmap 2022-11-27T15:14:42.241 INFO:tasks.ceph.osd.6.smithi085.stderr:2022-11-27T15:14:42.240+0000 7f470b5cc700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:14:42.466 DEBUG:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T15:14:42.592 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.582+0000 7fa87e313700 1 Processor -- start 2022-11-27T15:14:42.593 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.582+0000 7fa87e313700 1 -- start start 2022-11-27T15:14:42.593 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa87e313700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa87815cad0 0x7fa8780ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:42.594 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa87e313700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7fa8780b30a0 0x7fa8780b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:42.594 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa87e313700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8780b26e0 con 0x7fa8780b2c50 2022-11-27T15:14:42.594 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa87e313700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8780b3520 con 0x7fa8780b30a0 2022-11-27T15:14:42.594 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa87e313700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87815cea0 con 0x7fa87815cad0 2022-11-27T15:14:42.595 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa87c8b0700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7fa8780b30a0 0x7fa8780b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:42.595 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa8777fe700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7fa8780b2c50 0x7fa8780ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50882/0 (socket says 172.21.15.35:50882) 2022-11-27T15:14:42.595 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa8777fe700 1 -- 172.21.15.35:0/18677128 learned_addr learned my addr 172.21.15.35:0/18677128 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-27T15:14:42.595 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa877fff700 1 --2- 172.21.15.35:0/18677128 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa87815cad0 0x7fa8780ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:42.596 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa87c8b0700 1 -- 172.21.15.35:0/18677128 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa87815cad0 msgr2=0x7fa8780ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:42.596 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa87c8b0700 1 --2- 172.21.15.35:0/18677128 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa87815cad0 0x7fa8780ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:42.596 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa87c8b0700 1 -- 172.21.15.35:0/18677128 >> v1:172.21.15.35:6789/0 conn(0x7fa8780b2c50 legacy=0x7fa8780ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:42.596 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa87c8b0700 1 -- 172.21.15.35:0/18677128 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8780ca830 con 0x7fa8780b30a0 2022-11-27T15:14:42.597 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa876ffd700 1 -- 172.21.15.35:0/18677128 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530401661 0 0) 0x7fa8780b26e0 con 0x7fa8780b2c50 2022-11-27T15:14:42.597 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.583+0000 7fa877fff700 1 --2- 172.21.15.35:0/18677128 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa87815cad0 0x7fa8780ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T15:14:42.597 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa87c8b0700 1 --2- 172.21.15.35:0/18677128 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b30a0 0x7fa8780b1880 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa870009d90 tx=0x7fa87000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a381f6a6c4cb3489 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:42.597 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa876ffd700 1 -- 172.21.15.35:0/18677128 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000e070 con 0x7fa8780b30a0 2022-11-27T15:14:42.598 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa876ffd700 1 -- 172.21.15.35:0/18677128 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87000bbd0 con 0x7fa8780b30a0 2022-11-27T15:14:42.598 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa876ffd700 1 -- 172.21.15.35:0/18677128 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870012430 con 0x7fa8780b30a0 2022-11-27T15:14:42.598 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa87e313700 1 -- 172.21.15.35:0/18677128 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b30a0 msgr2=0x7fa8780b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:42.598 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa87e313700 1 --2- 172.21.15.35:0/18677128 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b30a0 0x7fa8780b1880 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa870009d90 tx=0x7fa87000b330 comp rx=0 tx=0).stop 2022-11-27T15:14:42.599 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa87e313700 1 -- 172.21.15.35:0/18677128 shutdown_connections 2022-11-27T15:14:42.599 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa87e313700 1 --2- 172.21.15.35:0/18677128 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b30a0 0x7fa8780b1880 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:42.599 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa87e313700 1 --2- 172.21.15.35:0/18677128 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa87815cad0 0x7fa8780ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:42.599 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa87e313700 1 -- 172.21.15.35:0/18677128 >> 172.21.15.35:0/18677128 conn(0x7fa87806df40 msgr2=0x7fa87806e310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:42.600 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa87e313700 1 -- 172.21.15.35:0/18677128 shutdown_connections 2022-11-27T15:14:42.600 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.584+0000 7fa87e313700 1 -- 172.21.15.35:0/18677128 wait complete. 2022-11-27T15:14:42.600 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.586+0000 7fa87e313700 1 Processor -- start 2022-11-27T15:14:42.600 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.586+0000 7fa87e313700 1 -- start start 2022-11-27T15:14:42.601 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa87e313700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b2c50 0x7fa8780c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:42.601 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa87e313700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8780b30a0 0x7fa8781483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:42.601 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa87e313700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa87815cad0 0x7fa8780c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:42.601 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa87e313700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780c4840 con 0x7fa8780b2c50 2022-11-27T15:14:42.602 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa87e313700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780c4b20 con 0x7fa87815cad0 2022-11-27T15:14:42.602 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa87e313700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780c4e00 con 0x7fa8780b30a0 2022-11-27T15:14:42.602 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa877fff700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b2c50 0x7fa8780c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:42.602 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa877fff700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b2c50 0x7fa8780c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39494/0 (socket says 172.21.15.35:39494) 2022-11-27T15:14:42.603 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa877fff700 1 -- 172.21.15.35:0/1592667009 learned_addr learned my addr 172.21.15.35:0/1592667009 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:42.603 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa8777fe700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8780b30a0 0x7fa8781483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:42.603 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa87c8b0700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa87815cad0 0x7fa8780c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:42.603 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa877fff700 1 -- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8780b30a0 msgr2=0x7fa8781483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:42.604 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa877fff700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8780b30a0 0x7fa8781483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:42.604 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa877fff700 1 -- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa87815cad0 msgr2=0x7fa8780c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:42.604 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa877fff700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa87815cad0 0x7fa8780c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:42.604 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.587+0000 7fa877fff700 1 -- 172.21.15.35:0/1592667009 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870009a70 con 0x7fa8780b2c50 2022-11-27T15:14:42.605 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.588+0000 7fa877fff700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b2c50 0x7fa8780c58e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e810 tx=0x7fa86000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:42.605 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.588+0000 7fa874ff9700 1 -- 172.21.15.35:0/1592667009 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000c020 con 0x7fa8780b2c50 2022-11-27T15:14:42.605 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.588+0000 7fa874ff9700 1 -- 172.21.15.35:0/1592667009 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860004250 con 0x7fa8780b2c50 2022-11-27T15:14:42.605 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.588+0000 7fa87e313700 1 -- 172.21.15.35:0/1592667009 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8780c50e0 con 0x7fa8780b2c50 2022-11-27T15:14:42.606 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.588+0000 7fa874ff9700 1 -- 172.21.15.35:0/1592667009 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600036a0 con 0x7fa8780b2c50 2022-11-27T15:14:42.606 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.588+0000 7fa87e313700 1 -- 172.21.15.35:0/1592667009 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8780c92f0 con 0x7fa8780b2c50 2022-11-27T15:14:42.606 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.589+0000 7fa87e313700 1 -- 172.21.15.35:0/1592667009 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87804e190 con 0x7fa8780b2c50 2022-11-27T15:14:42.606 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.589+0000 7fa874ff9700 1 -- 172.21.15.35:0/1592667009 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7fa860003800 con 0x7fa8780b2c50 2022-11-27T15:14:42.607 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.590+0000 7fa874ff9700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa85c066540 0x7fa85c0689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:42.607 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.590+0000 7fa874ff9700 1 -- 172.21.15.35:0/1592667009 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7fa8600824b0 con 0x7fa8780b2c50 2022-11-27T15:14:42.607 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.590+0000 7fa8777fe700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa85c066540 0x7fa85c0689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:42.608 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.590+0000 7fa8777fe700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa85c066540 0x7fa85c0689f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa8780cf1b0 tx=0x7fa868007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:42.608 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.594+0000 7fa874ff9700 1 -- 172.21.15.35:0/1592667009 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fa860050b00 con 0x7fa8780b2c50 2022-11-27T15:14:42.784 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.783+0000 7fa87e313700 1 -- 172.21.15.35:0/1592667009 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa87806eb80 con 0x7fa8780b2c50 2022-11-27T15:14:42.784 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.783+0000 7fa874ff9700 1 -- 172.21.15.35:0/1592667009 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7011 (secure 0 0 0) 0x7fa860054120 con 0x7fa8780b2c50 2022-11-27T15:14:42.785 INFO:teuthology.misc.health.smithi035.stdout: 2022-11-27T15:14:42.785 INFO:teuthology.misc.health.smithi035.stdout:{"epoch":12,"fsid":"1a1c2d0e-bda9-4965-9029-a4d82eabd38a","created":"2022-11-27T15:14:31.789352+0000","modified":"2022-11-27T15:14:40.146703+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T15:14:34.339079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"92e0fd37-a176-44da-847e-2bc153faa513","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e9828c4c-3cab-4b51-80fe-beaca70fd85a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"80ba487d-307a-46d3-bf27-fb037321414c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"962bb8b8-e6be-48ce-99c8-4697ba5fd364","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fd1b4ee8-0942-467c-b3cc-990bfb7c34c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f6133163-76e1-4748-83ad-e32cfaaaaf41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9bdcaeaa-8746-4eb9-bc3c-a3a5f3269550","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0aba1feb-9777-4a35-892d-f1ff9fec1d80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T15:14:42.787 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.785+0000 7fa87e313700 1 -- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa85c066540 msgr2=0x7fa85c0689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:42.787 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa85c066540 0x7fa85c0689f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa8780cf1b0 tx=0x7fa868007de0 comp rx=0 tx=0).stop 2022-11-27T15:14:42.787 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 -- 172.21.15.35:0/1592667009 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b2c50 msgr2=0x7fa8780c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:42.788 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b2c50 0x7fa8780c58e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e810 tx=0x7fa86000eb20 comp rx=0 tx=0).stop 2022-11-27T15:14:42.788 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 -- 172.21.15.35:0/1592667009 shutdown_connections 2022-11-27T15:14:42.788 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa85c066540 0x7fa85c0689f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:42.788 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa87815cad0 0x7fa8780c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:42.789 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa8780b2c50 0x7fa8780c58e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:42.789 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 --2- 172.21.15.35:0/1592667009 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8780b30a0 0x7fa8781483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:42.789 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 -- 172.21.15.35:0/1592667009 >> 172.21.15.35:0/1592667009 conn(0x7fa87806df40 msgr2=0x7fa87815a250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:42.789 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 -- 172.21.15.35:0/1592667009 shutdown_connections 2022-11-27T15:14:42.790 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:42.786+0000 7fa87e313700 1 -- 172.21.15.35:0/1592667009 wait complete. 2022-11-27T15:14:42.798 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-11-27T15:14:49.606 DEBUG:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T15:14:49.735 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.733+0000 7fa84f5b4700 1 Processor -- start 2022-11-27T15:14:49.736 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.734+0000 7fa84f5b4700 1 -- start start 2022-11-27T15:14:49.738 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.734+0000 7fa84f5b4700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7fa84815c880 0x7fa848163500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:49.738 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.734+0000 7fa84f5b4700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8480ca970 con 0x7fa848158640 2022-11-27T15:14:49.739 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.734+0000 7fa84f5b4700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa848158a10 con 0x7fa8480ce1f0 2022-11-27T15:14:49.739 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.734+0000 7fa84f5b4700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8480b0aa0 con 0x7fa84815c880 2022-11-27T15:14:49.739 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.735+0000 7fa84db51700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7fa84815c880 0x7fa848163500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:49.739 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.735+0000 7fa84d350700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7fa848158640 0x7fa8480cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:51070/0 (socket says 172.21.15.35:51070) 2022-11-27T15:14:49.740 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.735+0000 7fa84d350700 1 -- 172.21.15.35:0/1347585958 learned_addr learned my addr 172.21.15.35:0/1347585958 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-27T15:14:49.740 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.735+0000 7fa84db51700 1 -- 172.21.15.35:0/1347585958 >> v1:172.21.15.85:6789/0 conn(0x7fa8480ce1f0 legacy=0x7fa84815c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:49.740 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.735+0000 7fa84db51700 1 -- 172.21.15.35:0/1347585958 >> v1:172.21.15.35:6789/0 conn(0x7fa848158640 legacy=0x7fa8480cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:49.740 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.735+0000 7fa84db51700 1 -- 172.21.15.35:0/1347585958 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84815cda0 con 0x7fa84815c880 2022-11-27T15:14:49.740 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.735+0000 7fa84db51700 1 --2- 172.21.15.35:0/1347585958 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa84815c880 0x7fa848163500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa840009d90 tx=0x7fa84000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8af5a751dfe17b1 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:49.741 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.735+0000 7fa837fff700 1 -- 172.21.15.35:0/1347585958 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84000bdd0 con 0x7fa84815c880 2022-11-27T15:14:49.741 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.735+0000 7fa837fff700 1 -- 172.21.15.35:0/1347585958 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa840011460 con 0x7fa84815c880 2022-11-27T15:14:49.741 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.735+0000 7fa837fff700 1 -- 172.21.15.35:0/1347585958 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840011810 con 0x7fa84815c880 2022-11-27T15:14:49.741 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.736+0000 7fa84f5b4700 1 -- 172.21.15.35:0/1347585958 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa84815c880 msgr2=0x7fa848163500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:49.742 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.736+0000 7fa84f5b4700 1 --2- 172.21.15.35:0/1347585958 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa84815c880 0x7fa848163500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa840009d90 tx=0x7fa84000b330 comp rx=0 tx=0).stop 2022-11-27T15:14:49.742 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.736+0000 7fa84f5b4700 1 -- 172.21.15.35:0/1347585958 shutdown_connections 2022-11-27T15:14:49.742 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.736+0000 7fa84f5b4700 1 --2- 172.21.15.35:0/1347585958 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa84815c880 0x7fa848163500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:49.742 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.736+0000 7fa84f5b4700 1 -- 172.21.15.35:0/1347585958 >> 172.21.15.35:0/1347585958 conn(0x7fa84806b150 msgr2=0x7fa84806ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:49.743 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.736+0000 7fa84f5b4700 1 -- 172.21.15.35:0/1347585958 shutdown_connections 2022-11-27T15:14:49.743 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.736+0000 7fa84f5b4700 1 -- 172.21.15.35:0/1347585958 wait complete. 2022-11-27T15:14:49.746 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.737+0000 7fa84f5b4700 1 Processor -- start 2022-11-27T15:14:49.746 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.738+0000 7fa84f5b4700 1 -- start start 2022-11-27T15:14:49.747 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.738+0000 7fa84f5b4700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa848158640 0x7fa8480b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:49.747 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84f5b4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa84815c880 0x7fa848151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:49.747 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84f5b4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa848156670 0x7fa8480c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:49.748 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84f5b4700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848154b80 con 0x7fa848158640 2022-11-27T15:14:49.748 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84f5b4700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848154550 con 0x7fa84815c880 2022-11-27T15:14:49.748 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84f5b4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa848156c10 con 0x7fa848156670 2022-11-27T15:14:49.748 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84d350700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa848158640 0x7fa8480b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:49.749 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84d350700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa848158640 0x7fa8480b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39680/0 (socket says 172.21.15.35:39680) 2022-11-27T15:14:49.749 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84d350700 1 -- 172.21.15.35:0/958766807 learned_addr learned my addr 172.21.15.35:0/958766807 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:49.749 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84cb4f700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa84815c880 0x7fa848151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:49.749 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84d350700 1 -- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa848156670 msgr2=0x7fa8480c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:49.750 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84d350700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa848156670 0x7fa8480c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:49.750 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84d350700 1 -- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa84815c880 msgr2=0x7fa848151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:49.750 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84d350700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa84815c880 0x7fa848151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:49.750 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84d350700 1 -- 172.21.15.35:0/958766807 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840009a70 con 0x7fa848158640 2022-11-27T15:14:49.751 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84d350700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa848158640 0x7fa8480b3ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c700 tx=0x7fa83800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:49.751 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa835ffb700 1 -- 172.21.15.35:0/958766807 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8380074e0 con 0x7fa848158640 2022-11-27T15:14:49.752 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa835ffb700 1 -- 172.21.15.35:0/958766807 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa838007640 con 0x7fa848158640 2022-11-27T15:14:49.752 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa835ffb700 1 -- 172.21.15.35:0/958766807 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838005a30 con 0x7fa848158640 2022-11-27T15:14:49.753 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.739+0000 7fa84f5b4700 1 -- 172.21.15.35:0/958766807 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa848156ef0 con 0x7fa848158640 2022-11-27T15:14:49.753 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.740+0000 7fa84f5b4700 1 -- 172.21.15.35:0/958766807 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8480b41e0 con 0x7fa848158640 2022-11-27T15:14:49.753 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.740+0000 7fa84f5b4700 1 -- 172.21.15.35:0/958766807 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84804e190 con 0x7fa848158640 2022-11-27T15:14:49.754 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.741+0000 7fa835ffb700 1 -- 172.21.15.35:0/958766807 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fa838019030 con 0x7fa848158640 2022-11-27T15:14:49.754 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.741+0000 7fa835ffb700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa828066590 0x7fa828068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:49.755 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.741+0000 7fa835ffb700 1 -- 172.21.15.35:0/958766807 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa838082620 con 0x7fa848158640 2022-11-27T15:14:49.755 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.741+0000 7fa84cb4f700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa828066590 0x7fa828068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:49.756 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.742+0000 7fa84cb4f700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa828066590 0x7fa828068a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa848156430 tx=0x7fa844009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:49.757 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.746+0000 7fa835ffb700 1 -- 172.21.15.35:0/958766807 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fa83804abd0 con 0x7fa848158640 2022-11-27T15:14:49.939 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.937+0000 7fa84f5b4700 1 -- 172.21.15.35:0/958766807 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa84815d080 con 0x7fa848158640 2022-11-27T15:14:49.940 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.938+0000 7fa835ffb700 1 -- 172.21.15.35:0/958766807 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12173 (secure 0 0 0) 0x7fa838017090 con 0x7fa848158640 2022-11-27T15:14:49.940 INFO:teuthology.misc.health.smithi035.stdout: 2022-11-27T15:14:49.940 INFO:teuthology.misc.health.smithi035.stdout:{"epoch":16,"fsid":"1a1c2d0e-bda9-4965-9029-a4d82eabd38a","created":"2022-11-27T15:14:31.789352+0000","modified":"2022-11-27T15:14:46.043350+0000","last_up_change":"2022-11-27T15:14:43.038461+0000","last_in_change":"2022-11-27T15:14:34.339079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T15:14:43.200032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"92e0fd37-a176-44da-847e-2bc153faa513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":101387},{"type":"v1","addr":"172.21.15.35:6801","nonce":101387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":101387},{"type":"v1","addr":"172.21.15.35:6803","nonce":101387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":101387},{"type":"v1","addr":"172.21.15.35:6807","nonce":101387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":101387},{"type":"v1","addr":"172.21.15.35:6805","nonce":101387}]},"public_addr":"172.21.15.35:6801/101387","cluster_addr":"172.21.15.35:6803/101387","heartbeat_back_addr":"172.21.15.35:6807/101387","heartbeat_front_addr":"172.21.15.35:6805/101387","state":["exists","up"]},{"osd":1,"uuid":"e9828c4c-3cab-4b51-80fe-beaca70fd85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6816","nonce":101390},{"type":"v1","addr":"172.21.15.35:6817","nonce":101390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6818","nonce":101390},{"type":"v1","addr":"172.21.15.35:6819","nonce":101390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6822","nonce":101390},{"type":"v1","addr":"172.21.15.35:6823","nonce":101390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6820","nonce":101390},{"type":"v1","addr":"172.21.15.35:6821","nonce":101390}]},"public_addr":"172.21.15.35:6817/101390","cluster_addr":"172.21.15.35:6819/101390","heartbeat_back_addr":"172.21.15.35:6823/101390","heartbeat_front_addr":"172.21.15.35:6821/101390","state":["exists","up"]},{"osd":2,"uuid":"80ba487d-307a-46d3-bf27-fb037321414c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6808","nonce":101389},{"type":"v1","addr":"172.21.15.35:6809","nonce":101389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6810","nonce":101389},{"type":"v1","addr":"172.21.15.35:6811","nonce":101389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6814","nonce":101389},{"type":"v1","addr":"172.21.15.35:6815","nonce":101389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6812","nonce":101389},{"type":"v1","addr":"172.21.15.35:6813","nonce":101389}]},"public_addr":"172.21.15.35:6809/101389","cluster_addr":"172.21.15.35:6811/101389","heartbeat_back_addr":"172.21.15.35:6815/101389","heartbeat_front_addr":"172.21.15.35:6813/101389","state":["exists","up"]},{"osd":3,"uuid":"962bb8b8-e6be-48ce-99c8-4697ba5fd364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6824","nonce":101413},{"type":"v1","addr":"172.21.15.35:6825","nonce":101413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6826","nonce":101413},{"type":"v1","addr":"172.21.15.35:6827","nonce":101413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6830","nonce":101413},{"type":"v1","addr":"172.21.15.35:6831","nonce":101413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6828","nonce":101413},{"type":"v1","addr":"172.21.15.35:6829","nonce":101413}]},"public_addr":"172.21.15.35:6825/101413","cluster_addr":"172.21.15.35:6827/101413","heartbeat_back_addr":"172.21.15.35:6831/101413","heartbeat_front_addr":"172.21.15.35:6829/101413","state":["exists","up"]},{"osd":4,"uuid":"fd1b4ee8-0942-467c-b3cc-990bfb7c34c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":101623},{"type":"v1","addr":"172.21.15.85:6801","nonce":101623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":101623},{"type":"v1","addr":"172.21.15.85:6803","nonce":101623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":101623},{"type":"v1","addr":"172.21.15.85:6807","nonce":101623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":101623},{"type":"v1","addr":"172.21.15.85:6805","nonce":101623}]},"public_addr":"172.21.15.85:6801/101623","cluster_addr":"172.21.15.85:6803/101623","heartbeat_back_addr":"172.21.15.85:6807/101623","heartbeat_front_addr":"172.21.15.85:6805/101623","state":["exists","up"]},{"osd":5,"uuid":"f6133163-76e1-4748-83ad-e32cfaaaaf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":101624},{"type":"v1","addr":"172.21.15.85:6809","nonce":101624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":101624},{"type":"v1","addr":"172.21.15.85:6811","nonce":101624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":101624},{"type":"v1","addr":"172.21.15.85:6815","nonce":101624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":101624},{"type":"v1","addr":"172.21.15.85:6813","nonce":101624}]},"public_addr":"172.21.15.85:6809/101624","cluster_addr":"172.21.15.85:6811/101624","heartbeat_back_addr":"172.21.15.85:6815/101624","heartbeat_front_addr":"172.21.15.85:6813/101624","state":["exists","up"]},{"osd":6,"uuid":"9bdcaeaa-8746-4eb9-bc3c-a3a5f3269550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6816","nonce":101625},{"type":"v1","addr":"172.21.15.85:6817","nonce":101625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":101625},{"type":"v1","addr":"172.21.15.85:6819","nonce":101625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":101625},{"type":"v1","addr":"172.21.15.85:6823","nonce":101625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":101625},{"type":"v1","addr":"172.21.15.85:6821","nonce":101625}]},"public_addr":"172.21.15.85:6817/101625","cluster_addr":"172.21.15.85:6819/101625","heartbeat_back_addr":"172.21.15.85:6823/101625","heartbeat_front_addr":"172.21.15.85:6821/101625","state":["exists","up"]},{"osd":7,"uuid":"0aba1feb-9777-4a35-892d-f1ff9fec1d80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":101668},{"type":"v1","addr":"172.21.15.85:6825","nonce":101668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":101668},{"type":"v1","addr":"172.21.15.85:6827","nonce":101668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":101668},{"type":"v1","addr":"172.21.15.85:6831","nonce":101668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":101668},{"type":"v1","addr":"172.21.15.85:6829","nonce":101668}]},"public_addr":"172.21.15.85:6825/101668","cluster_addr":"172.21.15.85:6827/101668","heartbeat_back_addr":"172.21.15.85:6831/101668","heartbeat_front_addr":"172.21.15.85:6829/101668","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.290815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.534039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.488830+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.508610+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.233117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.549218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.502641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.594384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T15:14:49.943 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 -- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa828066590 msgr2=0x7fa828068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:49.943 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa828066590 0x7fa828068a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa848156430 tx=0x7fa844009040 comp rx=0 tx=0).stop 2022-11-27T15:14:49.943 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 -- 172.21.15.35:0/958766807 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa848158640 msgr2=0x7fa8480b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:49.943 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa848158640 0x7fa8480b3ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c700 tx=0x7fa83800cac0 comp rx=0 tx=0).stop 2022-11-27T15:14:49.944 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 -- 172.21.15.35:0/958766807 shutdown_connections 2022-11-27T15:14:49.944 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fa828066590 0x7fa828068a40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:49.944 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa848156670 0x7fa8480c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:49.944 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa848158640 0x7fa8480b3ca0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:49.945 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 --2- 172.21.15.35:0/958766807 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa84815c880 0x7fa848151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:49.945 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 -- 172.21.15.35:0/958766807 >> 172.21.15.35:0/958766807 conn(0x7fa8480ce1f0 msgr2=0x7fa848151ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:49.945 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 -- 172.21.15.35:0/958766807 shutdown_connections 2022-11-27T15:14:49.945 INFO:teuthology.misc.health.smithi035.stderr:2022-11-27T15:14:49.940+0000 7fa84f5b4700 1 -- 172.21.15.35:0/958766807 wait complete. 2022-11-27T15:14:49.953 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-11-27T15:14:49.953 INFO:tasks.ceph:Creating RBD pool 2022-11-27T15:14:49.954 DEBUG:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd pool create rbd 8 2022-11-27T15:14:50.077 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.075+0000 7f05f7a38700 1 Processor -- start 2022-11-27T15:14:50.078 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.075+0000 7f05f7a38700 1 -- start start 2022-11-27T15:14:50.083 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.076+0000 7f05f7a38700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015b5d0 0x7f05f00c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:50.083 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.076+0000 7f05f7a38700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f05f006aaf0 0x7f05f00cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:50.084 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.076+0000 7f05f7a38700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f0070780 con 0x7f05f015f3c0 2022-11-27T15:14:50.084 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.076+0000 7f05f7a38700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f0070a60 con 0x7f05f006aaf0 2022-11-27T15:14:50.084 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.076+0000 7f05f7a38700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f0070d40 con 0x7f05f015b5d0 2022-11-27T15:14:50.084 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.076+0000 7f05f4fd3700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f05f006aaf0 0x7f05f00cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:50.085 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.076+0000 7f05f5fd5700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f05f015f3c0 0x7f05f006dd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:51088/0 (socket says 172.21.15.35:51088) 2022-11-27T15:14:50.085 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.076+0000 7f05f5fd5700 1 -- 172.21.15.35:0/2250491340 learned_addr learned my addr 172.21.15.35:0/2250491340 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-27T15:14:50.085 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.076+0000 7f05f57d4700 1 --2- 172.21.15.35:0/2250491340 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015b5d0 0x7f05f00c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:50.085 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.077+0000 7f05f4fd3700 1 -- 172.21.15.35:0/2250491340 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015b5d0 msgr2=0x7f05f00c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:50.085 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.077+0000 7f05f4fd3700 1 --2- 172.21.15.35:0/2250491340 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015b5d0 0x7f05f00c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:50.086 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.077+0000 7f05f4fd3700 1 -- 172.21.15.35:0/2250491340 >> v1:172.21.15.35:6789/0 conn(0x7f05f015f3c0 legacy=0x7f05f006dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:50.086 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.077+0000 7f05dffff700 1 -- 172.21.15.35:0/2250491340 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437338550 0 0) 0x7f05f0070780 con 0x7f05f015f3c0 2022-11-27T15:14:50.086 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.077+0000 7f05f4fd3700 1 -- 172.21.15.35:0/2250491340 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f0071020 con 0x7f05f006aaf0 2022-11-27T15:14:50.086 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.077+0000 7f05f4fd3700 1 --2- 172.21.15.35:0/2250491340 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 0x7f05f00cbf60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009840 tx=0x7f05ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf147691549a1f76 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:50.087 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.077+0000 7f05dffff700 1 -- 172.21.15.35:0/2250491340 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec011830 con 0x7f05f006aaf0 2022-11-27T15:14:50.087 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.077+0000 7f05dffff700 1 -- 172.21.15.35:0/2250491340 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec011990 con 0x7f05f006aaf0 2022-11-27T15:14:50.087 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.077+0000 7f05dffff700 1 -- 172.21.15.35:0/2250491340 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec011d20 con 0x7f05f006aaf0 2022-11-27T15:14:50.087 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.078+0000 7f05f7a38700 1 -- 172.21.15.35:0/2250491340 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 msgr2=0x7f05f00cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:50.088 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.078+0000 7f05f7a38700 1 --2- 172.21.15.35:0/2250491340 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 0x7f05f00cbf60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009840 tx=0x7f05ec009b50 comp rx=0 tx=0).stop 2022-11-27T15:14:50.088 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.078+0000 7f05f7a38700 1 -- 172.21.15.35:0/2250491340 shutdown_connections 2022-11-27T15:14:50.088 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.078+0000 7f05f7a38700 1 --2- 172.21.15.35:0/2250491340 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015b5d0 0x7f05f00c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:50.088 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.078+0000 7f05f7a38700 1 --2- 172.21.15.35:0/2250491340 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 0x7f05f00cbf60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:50.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.078+0000 7f05f7a38700 1 -- 172.21.15.35:0/2250491340 >> 172.21.15.35:0/2250491340 conn(0x7f05f00aff10 msgr2=0x7f05f00b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:50.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.078+0000 7f05f7a38700 1 -- 172.21.15.35:0/2250491340 shutdown_connections 2022-11-27T15:14:50.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.078+0000 7f05f7a38700 1 -- 172.21.15.35:0/2250491340 wait complete. 2022-11-27T15:14:50.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.079+0000 7f05f7a38700 1 Processor -- start 2022-11-27T15:14:50.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.079+0000 7f05f7a38700 1 -- start start 2022-11-27T15:14:50.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f7a38700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 0x7f05f0157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:50.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f7a38700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05f015b5d0 0x7f05f00c2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:50.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f7a38700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015f3c0 0x7f05f00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:50.091 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f7a38700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f00c2570 con 0x7f05f006aaf0 2022-11-27T15:14:50.091 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f7a38700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f00c4870 con 0x7f05f015b5d0 2022-11-27T15:14:50.091 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f7a38700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f00c4b50 con 0x7f05f015f3c0 2022-11-27T15:14:50.091 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f57d4700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 0x7f05f0157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:50.092 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f57d4700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 0x7f05f0157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39694/0 (socket says 172.21.15.35:39694) 2022-11-27T15:14:50.092 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f57d4700 1 -- 172.21.15.35:0/3321344224 learned_addr learned my addr 172.21.15.35:0/3321344224 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:50.092 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f4fd3700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05f015b5d0 0x7f05f00c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:50.092 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f5fd5700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015f3c0 0x7f05f00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:50.092 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f57d4700 1 -- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015f3c0 msgr2=0x7f05f00ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:50.093 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f57d4700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015f3c0 0x7f05f00ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:50.093 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f57d4700 1 -- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05f015b5d0 msgr2=0x7f05f00c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:50.093 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f57d4700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05f015b5d0 0x7f05f00c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:50.093 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f57d4700 1 -- 172.21.15.35:0/3321344224 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0094a0 con 0x7f05f006aaf0 2022-11-27T15:14:50.094 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f4fd3700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05f015b5d0 0x7f05f00c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T15:14:50.094 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f5fd5700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015f3c0 0x7f05f00ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T15:14:50.094 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f57d4700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 0x7f05f0157ac0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f05e000e9b0 tx=0x7f05e000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:50.094 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05ddffb700 1 -- 172.21.15.35:0/3321344224 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0009710 con 0x7f05f006aaf0 2022-11-27T15:14:50.095 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05ddffb700 1 -- 172.21.15.35:0/3321344224 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e000c850 con 0x7f05f006aaf0 2022-11-27T15:14:50.095 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05f7a38700 1 -- 172.21.15.35:0/3321344224 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f00b1e00 con 0x7f05f006aaf0 2022-11-27T15:14:50.095 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.080+0000 7f05ddffb700 1 -- 172.21.15.35:0/3321344224 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000cbc0 con 0x7f05f006aaf0 2022-11-27T15:14:50.095 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.081+0000 7f05f7a38700 1 -- 172.21.15.35:0/3321344224 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f00c44b0 con 0x7f05f006aaf0 2022-11-27T15:14:50.095 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.081+0000 7f05f7a38700 1 -- 172.21.15.35:0/3321344224 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f00b2450 con 0x7f05f006aaf0 2022-11-27T15:14:50.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.082+0000 7f05ddffb700 1 -- 172.21.15.35:0/3321344224 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f05e0023070 con 0x7f05f006aaf0 2022-11-27T15:14:50.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.082+0000 7f05ddffb700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f05d0066590 0x7f05d0068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:50.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.082+0000 7f05ddffb700 1 -- 172.21.15.35:0/3321344224 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f05e0083690 con 0x7f05f006aaf0 2022-11-27T15:14:50.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.082+0000 7f05f4fd3700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f05d0066590 0x7f05d0068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:50.098 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.083+0000 7f05f4fd3700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f05d0066590 0x7f05d0068a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05f006af20 tx=0x7f05ec008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:50.098 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.087+0000 7f05ddffb700 1 -- 172.21.15.35:0/3321344224 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f05e0051250 con 0x7f05f006aaf0 2022-11-27T15:14:50.279 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:50.277+0000 7f05f7a38700 1 -- 172.21.15.35:0/3321344224 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f05f00b1c60 con 0x7f05f006aaf0 2022-11-27T15:14:51.080 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.078+0000 7f05ddffb700 1 -- 172.21.15.35:0/3321344224 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f05f00b1c60 con 0x7f05f006aaf0 2022-11-27T15:14:51.084 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.081+0000 7f05f7a38700 1 -- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f05d0066590 msgr2=0x7f05d0068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:51.084 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.081+0000 7f05f7a38700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f05d0066590 0x7f05d0068a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05f006af20 tx=0x7f05ec008030 comp rx=0 tx=0).stop 2022-11-27T15:14:51.084 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.081+0000 7f05f7a38700 1 -- 172.21.15.35:0/3321344224 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 msgr2=0x7f05f0157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:51.085 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.081+0000 7f05f7a38700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 0x7f05f0157ac0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f05e000e9b0 tx=0x7f05e000ed70 comp rx=0 tx=0).stop 2022-11-27T15:14:51.085 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.082+0000 7f05f7a38700 1 -- 172.21.15.35:0/3321344224 shutdown_connections 2022-11-27T15:14:51.085 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.082+0000 7f05f7a38700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f05d0066590 0x7f05d0068a40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:51.085 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.082+0000 7f05f7a38700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05f015f3c0 0x7f05f00ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:51.086 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.082+0000 7f05f7a38700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f05f006aaf0 0x7f05f0157ac0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:51.086 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.082+0000 7f05f7a38700 1 --2- 172.21.15.35:0/3321344224 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05f015b5d0 0x7f05f00c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:51.086 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.082+0000 7f05f7a38700 1 -- 172.21.15.35:0/3321344224 >> 172.21.15.35:0/3321344224 conn(0x7f05f00aff10 msgr2=0x7f05f0069410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:51.086 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.082+0000 7f05f7a38700 1 -- 172.21.15.35:0/3321344224 shutdown_connections 2022-11-27T15:14:51.086 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.082+0000 7f05f7a38700 1 -- 172.21.15.35:0/3321344224 wait complete. 2022-11-27T15:14:51.087 INFO:teuthology.orchestra.run.smithi035.stderr:pool 'rbd' created 2022-11-27T15:14:51.099 DEBUG:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-11-27T15:14:51.233 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.231+0000 7fbb8c9d7700 1 Processor -- start 2022-11-27T15:14:51.233 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.231+0000 7fbb8c9d7700 1 -- start start 2022-11-27T15:14:51.244 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8c9d7700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fbb8806aaf0 0x7fbb880c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:51.245 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8c9d7700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb88070780 con 0x7fbb8815b5d0 2022-11-27T15:14:51.245 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8c9d7700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb88070a60 con 0x7fbb880cc670 2022-11-27T15:14:51.245 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8c9d7700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb88070d40 con 0x7fbb8806aaf0 2022-11-27T15:14:51.246 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8659c700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7fbb8815b5d0 0x7fbb880cbf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:51098/0 (socket says 172.21.15.35:51098) 2022-11-27T15:14:51.246 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8659c700 1 -- 172.21.15.35:0/1954670118 learned_addr learned my addr 172.21.15.35:0/1954670118 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-27T15:14:51.246 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936770730 0 0) 0x7fbb88070780 con 0x7fbb8815b5d0 2022-11-27T15:14:51.246 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb86d9d700 1 --2- 172.21.15.35:0/1954670118 >> v2:172.21.15.85:3300/0 conn(0x7fbb8806aaf0 0x7fbb880c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:51.246 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb6c0035b0 con 0x7fbb8815b5d0 2022-11-27T15:14:51.247 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 <== mon.0 v1:172.21.15.35:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2132911952 0 0) 0x7fbb6c0035b0 con 0x7fbb8815b5d0 2022-11-27T15:14:51.247 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 >> v2:172.21.15.85:3300/0 conn(0x7fbb8806aaf0 msgr2=0x7fbb880c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:51.247 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8559a700 1 --2- 172.21.15.35:0/1954670118 >> v2:172.21.15.85:3300/0 conn(0x7fbb8806aaf0 0x7fbb880c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:51.247 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.232+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 >> v1:172.21.15.85:6789/0 conn(0x7fbb880cc670 legacy=0x7fbb8806dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:51.248 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 --> v1:172.21.15.35:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88071020 con 0x7fbb8815b5d0 2022-11-27T15:14:51.248 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 <== mon.0 v1:172.21.15.35:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1516710402 0 0) 0x7fbb7c001670 con 0x7fbb8815b5d0 2022-11-27T15:14:51.248 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 >> v1:172.21.15.35:6789/0 conn(0x7fbb8815b5d0 legacy=0x7fbb880cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:51.248 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb6c003a00 0x7fbb6c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:51.249 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb6c004850 0x7fbb6c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:51.249 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb6c005200 0x7fbb6c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:51.249 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb88070d40 con 0x7fbb6c005200 2022-11-27T15:14:51.249 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c00c080 con 0x7fbb6c003a00 2022-11-27T15:14:51.250 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c00c240 con 0x7fbb6c004850 2022-11-27T15:14:51.250 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb86d9d700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb6c005200 0x7fbb6c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:51.250 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8659c700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb6c003a00 0x7fbb6c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:51.250 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb85d9b700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb6c004850 0x7fbb6c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:51.250 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb86d9d700 1 -- 172.21.15.35:0/1954670118 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb6c004850 msgr2=0x7fbb6c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:51.251 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb86d9d700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb6c004850 0x7fbb6c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:51.251 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb86d9d700 1 -- 172.21.15.35:0/1954670118 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb6c003a00 msgr2=0x7fbb6c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:51.251 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb86d9d700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb6c003a00 0x7fbb6c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:51.251 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb86d9d700 1 -- 172.21.15.35:0/1954670118 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c0035b0 con 0x7fbb6c005200 2022-11-27T15:14:51.252 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb86d9d700 1 -- 172.21.15.35:0/1954670118 reap_dead start 2022-11-27T15:14:51.252 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb86d9d700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb6c005200 0x7fbb6c00ba10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009fc0 tx=0x7fbb6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cd6271b4bf3eaa server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:51.252 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80003a20 con 0x7fbb6c005200 2022-11-27T15:14:51.252 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8000f040 con 0x7fbb6c005200 2022-11-27T15:14:51.253 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.233+0000 7fbb8559a700 1 -- 172.21.15.35:0/1954670118 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80003de0 con 0x7fbb6c005200 2022-11-27T15:14:51.253 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.234+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/1954670118 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb6c005200 msgr2=0x7fbb6c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:51.253 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.234+0000 7fbb8c9d7700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb6c005200 0x7fbb6c00ba10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009fc0 tx=0x7fbb6c005880 comp rx=0 tx=0).stop 2022-11-27T15:14:51.253 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.234+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/1954670118 shutdown_connections 2022-11-27T15:14:51.254 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.234+0000 7fbb8c9d7700 1 --2- 172.21.15.35:0/1954670118 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb6c005200 0x7fbb6c00ba10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:51.254 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.234+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/1954670118 >> 172.21.15.35:0/1954670118 conn(0x7fbb880aff10 msgr2=0x7fbb880b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:51.254 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.234+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/1954670118 shutdown_connections 2022-11-27T15:14:51.254 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.234+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/1954670118 wait complete. 2022-11-27T15:14:51.254 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.235+0000 7fbb8c9d7700 1 Processor -- start 2022-11-27T15:14:51.255 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.235+0000 7fbb8c9d7700 1 -- start start 2022-11-27T15:14:51.255 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb8c9d7700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb6c005200 0x7fbb88157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:51.255 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb8c9d7700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb88159c00 0x7fbb8815a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:51.255 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb8c9d7700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb880c5460 0x7fbb880c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:51.256 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb8c9d7700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb880c5e40 con 0x7fbb880c5460 2022-11-27T15:14:51.256 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb8c9d7700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8815a5b0 con 0x7fbb88159c00 2022-11-27T15:14:51.256 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb8c9d7700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb880b3da0 con 0x7fbb6c005200 2022-11-27T15:14:51.256 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb86d9d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb880c5460 0x7fbb880c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:51.257 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb86d9d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb880c5460 0x7fbb880c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39718/0 (socket says 172.21.15.35:39718) 2022-11-27T15:14:51.257 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb86d9d700 1 -- 172.21.15.35:0/2114702469 learned_addr learned my addr 172.21.15.35:0/2114702469 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:51.257 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb85d9b700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb88159c00 0x7fbb8815a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:51.257 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb8659c700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb6c005200 0x7fbb88157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:51.258 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb86d9d700 1 -- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb6c005200 msgr2=0x7fbb88157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:51.258 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb86d9d700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb6c005200 0x7fbb88157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:51.258 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb86d9d700 1 -- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb88159c00 msgr2=0x7fbb8815a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:51.258 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb86d9d700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb88159c00 0x7fbb8815a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:51.259 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.236+0000 7fbb86d9d700 1 -- 172.21.15.35:0/2114702469 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb880c2b50 con 0x7fbb880c5460 2022-11-27T15:14:51.259 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.237+0000 7fbb86d9d700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb880c5460 0x7fbb880c5900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000e700 tx=0x7fbb80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:51.259 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.237+0000 7fbb76ffd700 1 -- 172.21.15.35:0/2114702469 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80003e10 con 0x7fbb880c5460 2022-11-27T15:14:51.259 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.237+0000 7fbb76ffd700 1 -- 172.21.15.35:0/2114702469 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb80004230 con 0x7fbb880c5460 2022-11-27T15:14:51.260 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.237+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/2114702469 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb880c6040 con 0x7fbb880c5460 2022-11-27T15:14:51.260 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.237+0000 7fbb76ffd700 1 -- 172.21.15.35:0/2114702469 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb800045e0 con 0x7fbb880c5460 2022-11-27T15:14:51.260 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.237+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/2114702469 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb880c6600 con 0x7fbb880c5460 2022-11-27T15:14:51.260 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.238+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/2114702469 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb880c2ce0 con 0x7fbb880c5460 2022-11-27T15:14:51.260 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.238+0000 7fbb76ffd700 1 -- 172.21.15.35:0/2114702469 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fbb80004740 con 0x7fbb880c5460 2022-11-27T15:14:51.261 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.239+0000 7fbb76ffd700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fbb64066590 0x7fbb64068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:51.261 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.239+0000 7fbb76ffd700 1 -- 172.21.15.35:0/2114702469 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fbb80085350 con 0x7fbb880c5460 2022-11-27T15:14:51.261 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.239+0000 7fbb8659c700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fbb64066590 0x7fbb64068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:51.261 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.240+0000 7fbb8659c700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fbb64066590 0x7fbb64068a40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c000c00 tx=0x7fbb7c007980 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:51.262 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.243+0000 7fbb76ffd700 1 -- 172.21.15.35:0/2114702469 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fbb80052e60 con 0x7fbb880c5460 2022-11-27T15:14:51.437 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:51.435+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/2114702469 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbb8804e190 con 0x7fbb880c5460 2022-11-27T15:14:52.083 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.082+0000 7fbb76ffd700 1 -- 172.21.15.35:0/2114702469 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7fbb80056480 con 0x7fbb880c5460 2022-11-27T15:14:52.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.085+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fbb64066590 msgr2=0x7fbb64068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.085+0000 7fbb8c9d7700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fbb64066590 0x7fbb64068a40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c000c00 tx=0x7fbb7c007980 comp rx=0 tx=0).stop 2022-11-27T15:14:52.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.085+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/2114702469 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb880c5460 msgr2=0x7fbb880c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.085+0000 7fbb8c9d7700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb880c5460 0x7fbb880c5900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000e700 tx=0x7fbb80009750 comp rx=0 tx=0).stop 2022-11-27T15:14:52.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.085+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/2114702469 shutdown_connections 2022-11-27T15:14:52.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.085+0000 7fbb8c9d7700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fbb64066590 0x7fbb64068a40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.091 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.085+0000 7fbb8c9d7700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fbb880c5460 0x7fbb880c5900 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.091 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.085+0000 7fbb8c9d7700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb6c005200 0x7fbb88157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.092 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.085+0000 7fbb8c9d7700 1 --2- 172.21.15.35:0/2114702469 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb88159c00 0x7fbb8815a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.092 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.085+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/2114702469 >> 172.21.15.35:0/2114702469 conn(0x7fbb880aff10 msgr2=0x7fbb880ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:52.092 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.086+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/2114702469 shutdown_connections 2022-11-27T15:14:52.093 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.086+0000 7fbb8c9d7700 1 -- 172.21.15.35:0/2114702469 wait complete. 2022-11-27T15:14:52.093 INFO:teuthology.orchestra.run.smithi035.stderr:enabled application 'rbd' on pool 'rbd' 2022-11-27T15:14:52.105 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-11-27T15:14:52.106 INFO:tasks.ceph.mds.a:Restarting daemon 2022-11-27T15:14:52.107 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-11-27T15:14:52.110 INFO:tasks.ceph.mds.a:Started 2022-11-27T15:14:52.111 INFO:tasks.ceph.mds.c:Restarting daemon 2022-11-27T15:14:52.111 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-11-27T15:14:52.114 INFO:tasks.ceph.mds.c:Started 2022-11-27T15:14:52.114 INFO:tasks.ceph.mds.b:Restarting daemon 2022-11-27T15:14:52.115 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-11-27T15:14:52.117 INFO:tasks.ceph.mds.b:Started 2022-11-27T15:14:52.118 INFO:tasks.ceph.mds.d:Restarting daemon 2022-11-27T15:14:52.118 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2022-11-27T15:14:52.120 INFO:tasks.ceph.mds.d:Started 2022-11-27T15:14:52.121 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-11-27T15:14:52.121 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T15:14:52.199 INFO:tasks.ceph.mds.a.smithi035.stdout:starting mds.a at 2022-11-27T15:14:52.200 INFO:tasks.ceph.mds.c.smithi035.stdout:starting mds.c at 2022-11-27T15:14:52.203 INFO:tasks.ceph.mds.d.smithi085.stdout:starting mds.d at 2022-11-27T15:14:52.204 INFO:tasks.ceph.mds.b.smithi085.stdout:starting mds.b at 2022-11-27T15:14:52.204 INFO:tasks.ceph.mds.d.smithi085.stderr:2022-11-27T15:14:52.203+0000 7faf874ca700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2022-11-27T15:14:52.260 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.258+0000 7fc5745d6700 1 Processor -- start 2022-11-27T15:14:52.261 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.258+0000 7fc5745d6700 1 -- start start 2022-11-27T15:14:52.266 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.259+0000 7fc5745d6700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7fc56c15b940 0x7fc56c160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.266 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.259+0000 7fc5745d6700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fc56c06cd50 0x7fc56c0709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.266 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.259+0000 7fc5745d6700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c15fb80 con 0x7fc56c070570 2022-11-27T15:14:52.267 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.259+0000 7fc5745d6700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c070fc0 con 0x7fc56c15b940 2022-11-27T15:14:52.267 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.259+0000 7fc5745d6700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c0712a0 con 0x7fc56c06cd50 2022-11-27T15:14:52.267 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.259+0000 7fc572372700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7fc56c15b940 0x7fc56c160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.267 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.259+0000 7fc572372700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7fc56c15b940 0x7fc56c160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39748/0 (socket says 172.21.15.35:39748) 2022-11-27T15:14:52.268 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.259+0000 7fc571b71700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7fc56c070570 0x7fc56c06c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:51144/0 (socket says 172.21.15.35:51144) 2022-11-27T15:14:52.268 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.259+0000 7fc572372700 1 -- 172.21.15.35:0/618625624 learned_addr learned my addr 172.21.15.35:0/618625624 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:52.268 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.260+0000 7fc572b73700 1 --2- 172.21.15.35:0/618625624 >> v2:172.21.15.85:3300/0 conn(0x7fc56c06cd50 0x7fc56c0709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.268 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.260+0000 7fc572372700 1 -- 172.21.15.35:0/618625624 >> v2:172.21.15.85:3300/0 conn(0x7fc56c06cd50 msgr2=0x7fc56c0709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:52.269 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.260+0000 7fc572372700 1 --2- 172.21.15.35:0/618625624 >> v2:172.21.15.85:3300/0 conn(0x7fc56c06cd50 0x7fc56c0709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.269 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.260+0000 7fc572372700 1 -- 172.21.15.35:0/618625624 >> v1:172.21.15.35:6789/0 conn(0x7fc56c070570 legacy=0x7fc56c06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.269 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.260+0000 7fc572372700 1 -- 172.21.15.35:0/618625624 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c15f530 con 0x7fc56c15b940 2022-11-27T15:14:52.269 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.260+0000 7fc571370700 1 -- 172.21.15.35:0/618625624 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628267703 0 0) 0x7fc56c15fb80 con 0x7fc56c070570 2022-11-27T15:14:52.269 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.260+0000 7fc572372700 1 --2- 172.21.15.35:0/618625624 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c15b940 0x7fc56c160010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc56c070500 tx=0x7fc560009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fc9b98f6a49954 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:52.270 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.260+0000 7fc571370700 1 -- 172.21.15.35:0/618625624 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc560011830 con 0x7fc56c15b940 2022-11-27T15:14:52.270 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.260+0000 7fc571370700 1 -- 172.21.15.35:0/618625624 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc560011990 con 0x7fc56c15b940 2022-11-27T15:14:52.270 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.260+0000 7fc571370700 1 -- 172.21.15.35:0/618625624 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc560011d20 con 0x7fc56c15b940 2022-11-27T15:14:52.270 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.261+0000 7fc5745d6700 1 -- 172.21.15.35:0/618625624 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c15b940 msgr2=0x7fc56c160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.271 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.261+0000 7fc5745d6700 1 --2- 172.21.15.35:0/618625624 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c15b940 0x7fc56c160010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc56c070500 tx=0x7fc560009bd0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.271 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.261+0000 7fc5745d6700 1 -- 172.21.15.35:0/618625624 shutdown_connections 2022-11-27T15:14:52.271 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.261+0000 7fc5745d6700 1 --2- 172.21.15.35:0/618625624 >> v2:172.21.15.85:3300/0 conn(0x7fc56c06cd50 0x7fc56c0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.271 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.261+0000 7fc5745d6700 1 --2- 172.21.15.35:0/618625624 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c15b940 0x7fc56c160010 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.272 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.261+0000 7fc5745d6700 1 -- 172.21.15.35:0/618625624 >> 172.21.15.35:0/618625624 conn(0x7fc56c0b0bb0 msgr2=0x7fc56c0ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:52.272 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.261+0000 7fc5745d6700 1 -- 172.21.15.35:0/618625624 shutdown_connections 2022-11-27T15:14:52.272 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.261+0000 7fc5745d6700 1 -- 172.21.15.35:0/618625624 wait complete. 2022-11-27T15:14:52.272 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.262+0000 7fc5745d6700 1 Processor -- start 2022-11-27T15:14:52.273 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.262+0000 7fc5745d6700 1 -- start start 2022-11-27T15:14:52.273 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc5745d6700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c06cd50 0x7fc56c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.274 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc5745d6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc56c070570 0x7fc56c0cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.274 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc5745d6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc56c15b940 0x7fc56c0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.274 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc5745d6700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0c69b0 con 0x7fc56c06cd50 2022-11-27T15:14:52.274 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc5745d6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0c6c90 con 0x7fc56c15b940 2022-11-27T15:14:52.275 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc5745d6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0c6f70 con 0x7fc56c070570 2022-11-27T15:14:52.275 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc572372700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c06cd50 0x7fc56c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.275 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc572372700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c06cd50 0x7fc56c15a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39754/0 (socket says 172.21.15.35:39754) 2022-11-27T15:14:52.275 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc572372700 1 -- 172.21.15.35:0/2544807118 learned_addr learned my addr 172.21.15.35:0/2544807118 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:52.276 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc572372700 1 -- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc56c070570 msgr2=0x7fc56c0cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.276 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc572372700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc56c070570 0x7fc56c0cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.276 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc572372700 1 -- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc56c15b940 msgr2=0x7fc56c0c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.276 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc572b73700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc56c15b940 0x7fc56c0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.277 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc572372700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc56c15b940 0x7fc56c0c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.277 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc572372700 1 -- 172.21.15.35:0/2544807118 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc560009550 con 0x7fc56c06cd50 2022-11-27T15:14:52.277 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc572372700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c06cd50 0x7fc56c15a800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc560009c10 tx=0x7fc560004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:52.277 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc55effd700 1 -- 172.21.15.35:0/2544807118 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5600036a0 con 0x7fc56c06cd50 2022-11-27T15:14:52.278 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc55effd700 1 -- 172.21.15.35:0/2544807118 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56001e070 con 0x7fc56c06cd50 2022-11-27T15:14:52.278 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc5745d6700 1 -- 172.21.15.35:0/2544807118 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c0c7250 con 0x7fc56c06cd50 2022-11-27T15:14:52.278 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.263+0000 7fc55effd700 1 -- 172.21.15.35:0/2544807118 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc560003a20 con 0x7fc56c06cd50 2022-11-27T15:14:52.278 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.264+0000 7fc5745d6700 1 -- 172.21.15.35:0/2544807118 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c15ad40 con 0x7fc56c06cd50 2022-11-27T15:14:52.278 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.264+0000 7fc5745d6700 1 -- 172.21.15.35:0/2544807118 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c15af30 con 0x7fc56c06cd50 2022-11-27T15:14:52.280 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.265+0000 7fc55effd700 1 -- 172.21.15.35:0/2544807118 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fc56000f050 con 0x7fc56c06cd50 2022-11-27T15:14:52.280 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.265+0000 7fc55effd700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc54c066590 0x7fc54c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.280 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.265+0000 7fc55effd700 1 -- 172.21.15.35:0/2544807118 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc560084490 con 0x7fc56c06cd50 2022-11-27T15:14:52.280 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.265+0000 7fc571b71700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc54c066590 0x7fc54c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.281 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.266+0000 7fc571b71700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc54c066590 0x7fc54c068a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc5680011e0 tx=0x7fc5680079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:52.281 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.270+0000 7fc55effd700 1 -- 172.21.15.35:0/2544807118 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fc560051ef0 con 0x7fc56c06cd50 2022-11-27T15:14:52.461 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.459+0000 7fc5745d6700 1 -- 172.21.15.35:0/2544807118 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc56c0c7770 con 0x7fc56c06cd50 2022-11-27T15:14:52.462 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.460+0000 7fc55effd700 1 -- 172.21.15.35:0/2544807118 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13706 (secure 0 0 0) 0x7fc560055510 con 0x7fc56c06cd50 2022-11-27T15:14:52.463 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:14:52.463 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":18,"fsid":"1a1c2d0e-bda9-4965-9029-a4d82eabd38a","created":"2022-11-27T15:14:31.789352+0000","modified":"2022-11-27T15:14:52.076692+0000","last_up_change":"2022-11-27T15:14:43.038461+0000","last_in_change":"2022-11-27T15:14:34.339079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T15:14:43.200032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T15:14:50.308558+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"92e0fd37-a176-44da-847e-2bc153faa513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":101387},{"type":"v1","addr":"172.21.15.35:6801","nonce":101387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":101387},{"type":"v1","addr":"172.21.15.35:6803","nonce":101387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":101387},{"type":"v1","addr":"172.21.15.35:6807","nonce":101387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":101387},{"type":"v1","addr":"172.21.15.35:6805","nonce":101387}]},"public_addr":"172.21.15.35:6801/101387","cluster_addr":"172.21.15.35:6803/101387","heartbeat_back_addr":"172.21.15.35:6807/101387","heartbeat_front_addr":"172.21.15.35:6805/101387","state":["exists","up"]},{"osd":1,"uuid":"e9828c4c-3cab-4b51-80fe-beaca70fd85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6816","nonce":101390},{"type":"v1","addr":"172.21.15.35:6817","nonce":101390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6818","nonce":101390},{"type":"v1","addr":"172.21.15.35:6819","nonce":101390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6822","nonce":101390},{"type":"v1","addr":"172.21.15.35:6823","nonce":101390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6820","nonce":101390},{"type":"v1","addr":"172.21.15.35:6821","nonce":101390}]},"public_addr":"172.21.15.35:6817/101390","cluster_addr":"172.21.15.35:6819/101390","heartbeat_back_addr":"172.21.15.35:6823/101390","heartbeat_front_addr":"172.21.15.35:6821/101390","state":["exists","up"]},{"osd":2,"uuid":"80ba487d-307a-46d3-bf27-fb037321414c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6808","nonce":101389},{"type":"v1","addr":"172.21.15.35:6809","nonce":101389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6810","nonce":101389},{"type":"v1","addr":"172.21.15.35:6811","nonce":101389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6814","nonce":101389},{"type":"v1","addr":"172.21.15.35:6815","nonce":101389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6812","nonce":101389},{"type":"v1","addr":"172.21.15.35:6813","nonce":101389}]},"public_addr":"172.21.15.35:6809/101389","cluster_addr":"172.21.15.35:6811/101389","heartbeat_back_addr":"172.21.15.35:6815/101389","heartbeat_front_addr":"172.21.15.35:6813/101389","state":["exists","up"]},{"osd":3,"uuid":"962bb8b8-e6be-48ce-99c8-4697ba5fd364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6824","nonce":101413},{"type":"v1","addr":"172.21.15.35:6825","nonce":101413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6826","nonce":101413},{"type":"v1","addr":"172.21.15.35:6827","nonce":101413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6830","nonce":101413},{"type":"v1","addr":"172.21.15.35:6831","nonce":101413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6828","nonce":101413},{"type":"v1","addr":"172.21.15.35:6829","nonce":101413}]},"public_addr":"172.21.15.35:6825/101413","cluster_addr":"172.21.15.35:6827/101413","heartbeat_back_addr":"172.21.15.35:6831/101413","heartbeat_front_addr":"172.21.15.35:6829/101413","state":["exists","up"]},{"osd":4,"uuid":"fd1b4ee8-0942-467c-b3cc-990bfb7c34c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":101623},{"type":"v1","addr":"172.21.15.85:6801","nonce":101623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":101623},{"type":"v1","addr":"172.21.15.85:6803","nonce":101623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":101623},{"type":"v1","addr":"172.21.15.85:6807","nonce":101623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":101623},{"type":"v1","addr":"172.21.15.85:6805","nonce":101623}]},"public_addr":"172.21.15.85:6801/101623","cluster_addr":"172.21.15.85:6803/101623","heartbeat_back_addr":"172.21.15.85:6807/101623","heartbeat_front_addr":"172.21.15.85:6805/101623","state":["exists","up"]},{"osd":5,"uuid":"f6133163-76e1-4748-83ad-e32cfaaaaf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":101624},{"type":"v1","addr":"172.21.15.85:6809","nonce":101624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":101624},{"type":"v1","addr":"172.21.15.85:6811","nonce":101624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":101624},{"type":"v1","addr":"172.21.15.85:6815","nonce":101624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":101624},{"type":"v1","addr":"172.21.15.85:6813","nonce":101624}]},"public_addr":"172.21.15.85:6809/101624","cluster_addr":"172.21.15.85:6811/101624","heartbeat_back_addr":"172.21.15.85:6815/101624","heartbeat_front_addr":"172.21.15.85:6813/101624","state":["exists","up"]},{"osd":6,"uuid":"9bdcaeaa-8746-4eb9-bc3c-a3a5f3269550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6816","nonce":101625},{"type":"v1","addr":"172.21.15.85:6817","nonce":101625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":101625},{"type":"v1","addr":"172.21.15.85:6819","nonce":101625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":101625},{"type":"v1","addr":"172.21.15.85:6823","nonce":101625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":101625},{"type":"v1","addr":"172.21.15.85:6821","nonce":101625}]},"public_addr":"172.21.15.85:6817/101625","cluster_addr":"172.21.15.85:6819/101625","heartbeat_back_addr":"172.21.15.85:6823/101625","heartbeat_front_addr":"172.21.15.85:6821/101625","state":["exists","up"]},{"osd":7,"uuid":"0aba1feb-9777-4a35-892d-f1ff9fec1d80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":101668},{"type":"v1","addr":"172.21.15.85:6825","nonce":101668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":101668},{"type":"v1","addr":"172.21.15.85:6827","nonce":101668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":101668},{"type":"v1","addr":"172.21.15.85:6831","nonce":101668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":101668},{"type":"v1","addr":"172.21.15.85:6829","nonce":101668}]},"public_addr":"172.21.15.85:6825/101668","cluster_addr":"172.21.15.85:6827/101668","heartbeat_back_addr":"172.21.15.85:6831/101668","heartbeat_front_addr":"172.21.15.85:6829/101668","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.290815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.534039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.488830+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.508610+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.233117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.549218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.502641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.594384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T15:14:52.465 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.462+0000 7fc5745d6700 1 -- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc54c066590 msgr2=0x7fc54c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.465 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.462+0000 7fc5745d6700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc54c066590 0x7fc54c068a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc5680011e0 tx=0x7fc5680079b0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.465 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.462+0000 7fc5745d6700 1 -- 172.21.15.35:0/2544807118 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c06cd50 msgr2=0x7fc56c15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.465 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.462+0000 7fc5745d6700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c06cd50 0x7fc56c15a800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc560009c10 tx=0x7fc560004750 comp rx=0 tx=0).stop 2022-11-27T15:14:52.466 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.462+0000 7fc5745d6700 1 -- 172.21.15.35:0/2544807118 shutdown_connections 2022-11-27T15:14:52.466 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.462+0000 7fc5745d6700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc54c066590 0x7fc54c068a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.466 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.462+0000 7fc5745d6700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc56c15b940 0x7fc56c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.466 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.462+0000 7fc5745d6700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc56c06cd50 0x7fc56c15a800 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.467 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.462+0000 7fc5745d6700 1 --2- 172.21.15.35:0/2544807118 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc56c070570 0x7fc56c0cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.467 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.462+0000 7fc5745d6700 1 -- 172.21.15.35:0/2544807118 >> 172.21.15.35:0/2544807118 conn(0x7fc56c0b0bb0 msgr2=0x7fc56c15be10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:52.467 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.463+0000 7fc5745d6700 1 -- 172.21.15.35:0/2544807118 shutdown_connections 2022-11-27T15:14:52.467 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.463+0000 7fc5745d6700 1 -- 172.21.15.35:0/2544807118 wait complete. 2022-11-27T15:14:52.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-11-27T15:14:43.200032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T15:14:50.308558+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T15:14:52.477 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T15:14:52.604 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.602+0000 7fc92094d700 1 Processor -- start 2022-11-27T15:14:52.605 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.602+0000 7fc92094d700 1 -- start start 2022-11-27T15:14:52.608 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.603+0000 7fc92094d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c0cc670 0x7fc91c0c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.608 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.603+0000 7fc92094d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fc91c06aaf0 0x7fc91c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.608 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.603+0000 7fc92094d700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c070780 con 0x7fc91c15b5d0 2022-11-27T15:14:52.609 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.603+0000 7fc92094d700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91c070a60 con 0x7fc91c06aaf0 2022-11-27T15:14:52.609 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.603+0000 7fc92094d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c070d40 con 0x7fc91c0cc670 2022-11-27T15:14:52.609 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.603+0000 7fc91a59c700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7fc91c15b5d0 0x7fc91c0cbf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:51158/0 (socket says 172.21.15.35:51158) 2022-11-27T15:14:52.609 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.603+0000 7fc91a59c700 1 -- 172.21.15.35:0/1952466358 learned_addr learned my addr 172.21.15.35:0/1952466358 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-27T15:14:52.610 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc919d9b700 1 --2- 172.21.15.35:0/1952466358 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c0cc670 0x7fc91c0c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.610 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91ad9d700 1 --2- 172.21.15.35:0/1952466358 >> v2:172.21.15.85:3300/0 conn(0x7fc91c06aaf0 0x7fc91c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.610 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91959a700 1 -- 172.21.15.35:0/1952466358 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269559900 0 0) 0x7fc91c070780 con 0x7fc91c15b5d0 2022-11-27T15:14:52.610 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91959a700 1 -- 172.21.15.35:0/1952466358 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9000035b0 con 0x7fc91c15b5d0 2022-11-27T15:14:52.611 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91ad9d700 1 -- 172.21.15.35:0/1952466358 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c0cc670 msgr2=0x7fc91c0c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:52.611 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91ad9d700 1 --2- 172.21.15.35:0/1952466358 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c0cc670 0x7fc91c0c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.611 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91ad9d700 1 -- 172.21.15.35:0/1952466358 >> v1:172.21.15.35:6789/0 conn(0x7fc91c15b5d0 legacy=0x7fc91c0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.611 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91ad9d700 1 -- 172.21.15.35:0/1952466358 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c071020 con 0x7fc91c06aaf0 2022-11-27T15:14:52.611 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc919d9b700 1 --2- 172.21.15.35:0/1952466358 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c0cc670 0x7fc91c0c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:52.612 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91ad9d700 1 --2- 172.21.15.35:0/1952466358 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc91c06aaf0 0x7fc91c06dd20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc914009d90 tx=0x7fc91400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c8a6c3d15672135 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:52.612 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91959a700 1 -- 172.21.15.35:0/1952466358 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91400e070 con 0x7fc91c06aaf0 2022-11-27T15:14:52.612 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91959a700 1 -- 172.21.15.35:0/1952466358 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91400bb20 con 0x7fc91c06aaf0 2022-11-27T15:14:52.612 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.604+0000 7fc91959a700 1 -- 172.21.15.35:0/1952466358 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914012430 con 0x7fc91c06aaf0 2022-11-27T15:14:52.613 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.605+0000 7fc92094d700 1 -- 172.21.15.35:0/1952466358 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc91c06aaf0 msgr2=0x7fc91c06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.613 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.605+0000 7fc92094d700 1 --2- 172.21.15.35:0/1952466358 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc91c06aaf0 0x7fc91c06dd20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc914009d90 tx=0x7fc91400b330 comp rx=0 tx=0).stop 2022-11-27T15:14:52.613 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.605+0000 7fc92094d700 1 -- 172.21.15.35:0/1952466358 shutdown_connections 2022-11-27T15:14:52.613 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.605+0000 7fc92094d700 1 --2- 172.21.15.35:0/1952466358 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc91c06aaf0 0x7fc91c06dd20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.614 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.605+0000 7fc92094d700 1 --2- 172.21.15.35:0/1952466358 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c0cc670 0x7fc91c0c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.614 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.605+0000 7fc92094d700 1 -- 172.21.15.35:0/1952466358 >> 172.21.15.35:0/1952466358 conn(0x7fc91c0aff10 msgr2=0x7fc91c0b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:52.614 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.605+0000 7fc92094d700 1 -- 172.21.15.35:0/1952466358 shutdown_connections 2022-11-27T15:14:52.614 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.605+0000 7fc92094d700 1 -- 172.21.15.35:0/1952466358 wait complete. 2022-11-27T15:14:52.614 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.606+0000 7fc92094d700 1 Processor -- start 2022-11-27T15:14:52.615 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.606+0000 7fc92094d700 1 -- start start 2022-11-27T15:14:52.618 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc92094d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c06aaf0 0x7fc91c1f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.618 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc92094d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc91c0cc670 0x7fc91c0b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.619 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc92094d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc91c15b5d0 0x7fc91c0aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.619 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc92094d700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0b3500 con 0x7fc91c15b5d0 2022-11-27T15:14:52.619 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc92094d700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c1f1b50 con 0x7fc91c0cc670 2022-11-27T15:14:52.620 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc92094d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0b2b80 con 0x7fc91c06aaf0 2022-11-27T15:14:52.620 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc91ad9d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc91c15b5d0 0x7fc91c0aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.620 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc91ad9d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc91c15b5d0 0x7fc91c0aca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39772/0 (socket says 172.21.15.35:39772) 2022-11-27T15:14:52.620 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc91ad9d700 1 -- 172.21.15.35:0/1063554085 learned_addr learned my addr 172.21.15.35:0/1063554085 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:52.621 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc91a59c700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c06aaf0 0x7fc91c1f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.621 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc919d9b700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc91c0cc670 0x7fc91c0b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.621 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc91ad9d700 1 -- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c06aaf0 msgr2=0x7fc91c1f1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.621 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc91ad9d700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c06aaf0 0x7fc91c1f1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc91ad9d700 1 -- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc91c0cc670 msgr2=0x7fc91c0b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc91ad9d700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc91c0cc670 0x7fc91c0b3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.607+0000 7fc91ad9d700 1 -- 172.21.15.35:0/1063554085 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c0094a0 con 0x7fc91c15b5d0 2022-11-27T15:14:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.608+0000 7fc91ad9d700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc91c15b5d0 0x7fc91c0aca80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc914013000 tx=0x7fc914004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:52.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.608+0000 7fc90affd700 1 -- 172.21.15.35:0/1063554085 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140129f0 con 0x7fc91c15b5d0 2022-11-27T15:14:52.623 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.608+0000 7fc90affd700 1 -- 172.21.15.35:0/1063554085 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc914020070 con 0x7fc91c15b5d0 2022-11-27T15:14:52.623 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.608+0000 7fc92094d700 1 -- 172.21.15.35:0/1063554085 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc914009a70 con 0x7fc91c15b5d0 2022-11-27T15:14:52.623 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.608+0000 7fc90affd700 1 -- 172.21.15.35:0/1063554085 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914012db0 con 0x7fc91c15b5d0 2022-11-27T15:14:52.623 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.608+0000 7fc92094d700 1 -- 172.21.15.35:0/1063554085 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91c0b2e40 con 0x7fc91c15b5d0 2022-11-27T15:14:52.624 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.609+0000 7fc92094d700 1 -- 172.21.15.35:0/1063554085 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91c0b3640 con 0x7fc91c15b5d0 2022-11-27T15:14:52.624 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.609+0000 7fc90affd700 1 -- 172.21.15.35:0/1063554085 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fc914004040 con 0x7fc91c15b5d0 2022-11-27T15:14:52.624 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.609+0000 7fc90affd700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc8f80665c0 0x7fc8f8068a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.625 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.610+0000 7fc90affd700 1 -- 172.21.15.35:0/1063554085 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc914022070 con 0x7fc91c15b5d0 2022-11-27T15:14:52.625 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.610+0000 7fc91a59c700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc8f80665c0 0x7fc8f8068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.625 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.610+0000 7fc91a59c700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc8f80665c0 0x7fc8f8068a70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc910007fd0 tx=0x7fc910007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:52.625 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.614+0000 7fc90affd700 1 -- 172.21.15.35:0/1063554085 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fc91c0b3640 con 0x7fc91c15b5d0 2022-11-27T15:14:52.806 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.803+0000 7fc92094d700 1 -- 172.21.15.35:0/1063554085 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc91c0b3640 con 0x7fc91c15b5d0 2022-11-27T15:14:52.806 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.804+0000 7fc90affd700 1 -- 172.21.15.35:0/1063554085 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc91c0b3640 con 0x7fc91c15b5d0 2022-11-27T15:14:52.807 INFO:teuthology.orchestra.run.smithi035.stdout:pg_num: 1 2022-11-27T15:14:52.809 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.805+0000 7fc92094d700 1 -- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc8f80665c0 msgr2=0x7fc8f8068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.810 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc8f80665c0 0x7fc8f8068a70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc910007fd0 tx=0x7fc910007dc0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.810 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 -- 172.21.15.35:0/1063554085 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc91c15b5d0 msgr2=0x7fc91c0aca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.811 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc91c15b5d0 0x7fc91c0aca80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc914013000 tx=0x7fc914004ad0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.812 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 -- 172.21.15.35:0/1063554085 shutdown_connections 2022-11-27T15:14:52.812 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fc8f80665c0 0x7fc8f8068a70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.812 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc91c15b5d0 0x7fc91c0aca80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.813 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc91c06aaf0 0x7fc91c1f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.813 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 --2- 172.21.15.35:0/1063554085 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc91c0cc670 0x7fc91c0b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.814 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 -- 172.21.15.35:0/1063554085 >> 172.21.15.35:0/1063554085 conn(0x7fc91c0aff10 msgr2=0x7fc91c0c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:52.814 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 -- 172.21.15.35:0/1063554085 shutdown_connections 2022-11-27T15:14:52.815 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.806+0000 7fc92094d700 1 -- 172.21.15.35:0/1063554085 wait complete. 2022-11-27T15:14:52.820 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T15:14:52.951 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.946+0000 7f0b44e7e700 1 Processor -- start 2022-11-27T15:14:52.952 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.946+0000 7f0b44e7e700 1 -- start start 2022-11-27T15:14:52.952 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b44e7e700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f0b400cc670 0x7f0b400c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.952 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b44e7e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4006aaf0 0x7f0b4006dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.953 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b44e7e700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b40070780 con 0x7f0b4015b5d0 2022-11-27T15:14:52.953 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b44e7e700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b40070a60 con 0x7f0b400cc670 2022-11-27T15:14:52.953 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b44e7e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b40070d40 con 0x7f0b4006aaf0 2022-11-27T15:14:52.953 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b3dd9b700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f0b400cc670 0x7f0b400c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.953 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b3dd9b700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f0b400cc670 0x7f0b400c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39778/0 (socket says 172.21.15.35:39778) 2022-11-27T15:14:52.954 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b3e59c700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f0b4015b5d0 0x7f0b400cbf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:51172/0 (socket says 172.21.15.35:51172) 2022-11-27T15:14:52.954 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b3dd9b700 1 -- 172.21.15.35:0/828756029 learned_addr learned my addr 172.21.15.35:0/828756029 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:52.955 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b3ed9d700 1 --2- 172.21.15.35:0/828756029 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4006aaf0 0x7f0b4006dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.955 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b3dd9b700 1 -- 172.21.15.35:0/828756029 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4006aaf0 msgr2=0x7f0b4006dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:52.955 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b3dd9b700 1 --2- 172.21.15.35:0/828756029 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4006aaf0 0x7f0b4006dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.955 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b3dd9b700 1 -- 172.21.15.35:0/828756029 >> v1:172.21.15.35:6789/0 conn(0x7f0b4015b5d0 legacy=0x7f0b400cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.956 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b3dd9b700 1 -- 172.21.15.35:0/828756029 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b40071020 con 0x7f0b400cc670 2022-11-27T15:14:52.956 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.947+0000 7f0b3dd9b700 1 --2- 172.21.15.35:0/828756029 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b400cc670 0x7f0b400c89e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0b300098f0 tx=0x7f0b30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f128c0b81069a6b1 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:52.956 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b3d59a700 1 -- 172.21.15.35:0/828756029 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000b8a0 con 0x7f0b400cc670 2022-11-27T15:14:52.956 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b3d59a700 1 -- 172.21.15.35:0/828756029 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3000ba00 con 0x7f0b400cc670 2022-11-27T15:14:52.957 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b3d59a700 1 -- 172.21.15.35:0/828756029 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000bd90 con 0x7f0b400cc670 2022-11-27T15:14:52.957 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b44e7e700 1 -- 172.21.15.35:0/828756029 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b400cc670 msgr2=0x7f0b400c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.957 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b44e7e700 1 --2- 172.21.15.35:0/828756029 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b400cc670 0x7f0b400c89e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0b300098f0 tx=0x7f0b30009c00 comp rx=0 tx=0).stop 2022-11-27T15:14:52.957 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b44e7e700 1 -- 172.21.15.35:0/828756029 shutdown_connections 2022-11-27T15:14:52.958 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b44e7e700 1 --2- 172.21.15.35:0/828756029 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4006aaf0 0x7f0b4006dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.958 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b44e7e700 1 --2- 172.21.15.35:0/828756029 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b400cc670 0x7f0b400c89e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.958 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b44e7e700 1 -- 172.21.15.35:0/828756029 >> 172.21.15.35:0/828756029 conn(0x7f0b400aff10 msgr2=0x7f0b400b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:52.958 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b44e7e700 1 -- 172.21.15.35:0/828756029 shutdown_connections 2022-11-27T15:14:52.959 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.948+0000 7f0b44e7e700 1 -- 172.21.15.35:0/828756029 wait complete. 2022-11-27T15:14:52.966 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.950+0000 7f0b44e7e700 1 Processor -- start 2022-11-27T15:14:52.966 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.950+0000 7f0b44e7e700 1 -- start start 2022-11-27T15:14:52.967 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b44e7e700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b4006aaf0 0x7f0b400b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.967 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b44e7e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b400cc670 0x7f0b401535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.968 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b44e7e700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b4015b5d0 0x7f0b400c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.968 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b44e7e700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400b3700 con 0x7f0b4015b5d0 2022-11-27T15:14:52.968 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b44e7e700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b40153b20 con 0x7f0b4006aaf0 2022-11-27T15:14:52.969 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b44e7e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400b2a80 con 0x7f0b400cc670 2022-11-27T15:14:52.969 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b3ed9d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b4015b5d0 0x7f0b400c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.970 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b3ed9d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b4015b5d0 0x7f0b400c2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39786/0 (socket says 172.21.15.35:39786) 2022-11-27T15:14:52.970 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b3ed9d700 1 -- 172.21.15.35:0/4214998 learned_addr learned my addr 172.21.15.35:0/4214998 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:52.970 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b3e59c700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b4006aaf0 0x7f0b400b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.971 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b3dd9b700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b400cc670 0x7f0b401535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.971 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b3ed9d700 1 -- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b400cc670 msgr2=0x7f0b401535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.971 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b3ed9d700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b400cc670 0x7f0b401535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.972 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b3ed9d700 1 -- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b4006aaf0 msgr2=0x7f0b400b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:52.972 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b3ed9d700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b4006aaf0 0x7f0b400b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:52.972 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.951+0000 7f0b3ed9d700 1 -- 172.21.15.35:0/4214998 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b30009550 con 0x7f0b4015b5d0 2022-11-27T15:14:52.972 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.952+0000 7f0b3ed9d700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b4015b5d0 0x7f0b400c2f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b38007e40 tx=0x7f0b3800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:52.973 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.952+0000 7f0b2effd700 1 -- 172.21.15.35:0/4214998 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3800ecd0 con 0x7f0b4015b5d0 2022-11-27T15:14:52.973 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.952+0000 7f0b2effd700 1 -- 172.21.15.35:0/4214998 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3800ee50 con 0x7f0b4015b5d0 2022-11-27T15:14:52.973 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.952+0000 7f0b44e7e700 1 -- 172.21.15.35:0/4214998 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b400b2d60 con 0x7f0b4015b5d0 2022-11-27T15:14:52.974 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.952+0000 7f0b2effd700 1 -- 172.21.15.35:0/4214998 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38014660 con 0x7f0b4015b5d0 2022-11-27T15:14:52.974 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.952+0000 7f0b44e7e700 1 -- 172.21.15.35:0/4214998 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b400b3320 con 0x7f0b4015b5d0 2022-11-27T15:14:52.974 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.953+0000 7f0b44e7e700 1 -- 172.21.15.35:0/4214998 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b400b3840 con 0x7f0b4015b5d0 2022-11-27T15:14:52.974 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.953+0000 7f0b2effd700 1 -- 172.21.15.35:0/4214998 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f0b38012050 con 0x7f0b4015b5d0 2022-11-27T15:14:52.974 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.953+0000 7f0b2effd700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0b1c066590 0x7f0b1c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:52.975 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.954+0000 7f0b2effd700 1 -- 172.21.15.35:0/4214998 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0b380839c0 con 0x7f0b4015b5d0 2022-11-27T15:14:52.975 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.954+0000 7f0b3e59c700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0b1c066590 0x7f0b1c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:52.975 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.954+0000 7f0b3e59c700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0b1c066590 0x7f0b1c068a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0b34000f60 tx=0x7f0b34007ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:52.975 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:52.958+0000 7f0b2effd700 1 -- 172.21.15.35:0/4214998 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f0b38051420 con 0x7f0b4015b5d0 2022-11-27T15:14:53.149 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.147+0000 7f0b44e7e700 1 -- 172.21.15.35:0/4214998 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0b4004e190 con 0x7f0b4015b5d0 2022-11-27T15:14:53.150 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.148+0000 7f0b2effd700 1 -- 172.21.15.35:0/4214998 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0b38054a40 con 0x7f0b4015b5d0 2022-11-27T15:14:53.151 INFO:teuthology.orchestra.run.smithi035.stdout:pg_num: 8 2022-11-27T15:14:53.152 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 -- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0b1c066590 msgr2=0x7f0b1c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.154 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0b1c066590 0x7f0b1c068a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0b34000f60 tx=0x7f0b34007ed0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.154 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 -- 172.21.15.35:0/4214998 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b4015b5d0 msgr2=0x7f0b400c2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.154 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b4015b5d0 0x7f0b400c2f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b38007e40 tx=0x7f0b3800e3c0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.155 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 -- 172.21.15.35:0/4214998 shutdown_connections 2022-11-27T15:14:53.155 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0b1c066590 0x7f0b1c068a40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.155 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0b4015b5d0 0x7f0b400c2f30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.155 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b4006aaf0 0x7f0b400b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 --2- 172.21.15.35:0/4214998 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b400cc670 0x7f0b401535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 -- 172.21.15.35:0/4214998 >> 172.21.15.35:0/4214998 conn(0x7f0b400aff10 msgr2=0x7f0b400af220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:53.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 -- 172.21.15.35:0/4214998 shutdown_connections 2022-11-27T15:14:53.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.150+0000 7f0b44e7e700 1 -- 172.21.15.35:0/4214998 wait complete. 2022-11-27T15:14:53.165 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T15:14:53.289 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.287+0000 7fe08389a700 1 Processor -- start 2022-11-27T15:14:53.290 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.288+0000 7fe08389a700 1 -- start start 2022-11-27T15:14:53.292 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe08389a700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fe07c0cbff0 0x7fe07c0ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:53.293 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe08389a700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe07c159b10 con 0x7fe07c0b17b0 2022-11-27T15:14:53.293 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe08389a700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe07c0ca380 con 0x7fe07c15dbc0 2022-11-27T15:14:53.293 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe08389a700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07c0cdba0 con 0x7fe07c0cbff0 2022-11-27T15:14:53.293 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe081636700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7fe07c0b17b0 0x7fe07c0b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:51188/0 (socket says 172.21.15.35:51188) 2022-11-27T15:14:53.294 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe081636700 1 -- 172.21.15.35:0/1046700204 learned_addr learned my addr 172.21.15.35:0/1046700204 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-27T15:14:53.294 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe081e37700 1 --2- 172.21.15.35:0/1046700204 >> v2:172.21.15.85:3300/0 conn(0x7fe07c0cbff0 0x7fe07c0ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:53.294 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe06bfff700 1 -- 172.21.15.35:0/1046700204 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477295339 0 0) 0x7fe078001110 con 0x7fe07c0b17b0 2022-11-27T15:14:53.294 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe06bfff700 1 -- 172.21.15.35:0/1046700204 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0640035b0 con 0x7fe07c0b17b0 2022-11-27T15:14:53.295 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe081e37700 1 -- 172.21.15.35:0/1046700204 >> v1:172.21.15.85:6789/0 conn(0x7fe07c15dbc0 legacy=0x7fe07c0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.295 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe081e37700 1 -- 172.21.15.35:0/1046700204 >> v1:172.21.15.35:6789/0 conn(0x7fe07c0b17b0 legacy=0x7fe07c0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.295 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.289+0000 7fe081e37700 1 -- 172.21.15.35:0/1046700204 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c0af530 con 0x7fe07c0cbff0 2022-11-27T15:14:53.295 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe081e37700 1 --2- 172.21.15.35:0/1046700204 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe07c0cbff0 0x7fe07c0ce6d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe074009d90 tx=0x7fe07400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e61441025f228eb7 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:53.296 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe06bfff700 1 -- 172.21.15.35:0/1046700204 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07400e070 con 0x7fe07c0cbff0 2022-11-27T15:14:53.296 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe06bfff700 1 -- 172.21.15.35:0/1046700204 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07400bb20 con 0x7fe07c0cbff0 2022-11-27T15:14:53.296 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe06bfff700 1 -- 172.21.15.35:0/1046700204 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074012430 con 0x7fe07c0cbff0 2022-11-27T15:14:53.296 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe08389a700 1 -- 172.21.15.35:0/1046700204 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe07c0cbff0 msgr2=0x7fe07c0ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.296 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe08389a700 1 --2- 172.21.15.35:0/1046700204 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe07c0cbff0 0x7fe07c0ce6d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe074009d90 tx=0x7fe07400b330 comp rx=0 tx=0).stop 2022-11-27T15:14:53.297 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe08389a700 1 -- 172.21.15.35:0/1046700204 shutdown_connections 2022-11-27T15:14:53.297 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe08389a700 1 --2- 172.21.15.35:0/1046700204 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe07c0cbff0 0x7fe07c0ce6d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.297 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe08389a700 1 -- 172.21.15.35:0/1046700204 >> 172.21.15.35:0/1046700204 conn(0x7fe07c06eb60 msgr2=0x7fe07c0cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:53.297 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe08389a700 1 -- 172.21.15.35:0/1046700204 shutdown_connections 2022-11-27T15:14:53.298 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.290+0000 7fe08389a700 1 -- 172.21.15.35:0/1046700204 wait complete. 2022-11-27T15:14:53.301 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.292+0000 7fe08389a700 1 Processor -- start 2022-11-27T15:14:53.301 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.292+0000 7fe08389a700 1 -- start start 2022-11-27T15:14:53.302 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe08389a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe07c0b17b0 0x7fe07c14c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:53.302 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe08389a700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe07c0cbff0 0x7fe07c0affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:53.302 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe08389a700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe07c15dbc0 0x7fe07c0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:53.302 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe08389a700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c0c7d00 con 0x7fe07c15dbc0 2022-11-27T15:14:53.302 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe08389a700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c0c7fe0 con 0x7fe07c0cbff0 2022-11-27T15:14:53.303 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe08389a700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c0c82c0 con 0x7fe07c0b17b0 2022-11-27T15:14:53.303 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe081e37700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe07c15dbc0 0x7fe07c0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:53.303 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe081e37700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe07c15dbc0 0x7fe07c0b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39802/0 (socket says 172.21.15.35:39802) 2022-11-27T15:14:53.303 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe081e37700 1 -- 172.21.15.35:0/2902098828 learned_addr learned my addr 172.21.15.35:0/2902098828 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:53.304 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe080e35700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe07c0cbff0 0x7fe07c0affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:53.304 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe081636700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe07c0b17b0 0x7fe07c14c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:53.304 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe081e37700 1 -- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe07c0b17b0 msgr2=0x7fe07c14c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.304 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe081e37700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe07c0b17b0 0x7fe07c14c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.305 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe081e37700 1 -- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe07c0cbff0 msgr2=0x7fe07c0affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.305 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe081e37700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe07c0cbff0 0x7fe07c0affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.305 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe081e37700 1 -- 172.21.15.35:0/2902098828 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe074009a70 con 0x7fe07c15dbc0 2022-11-27T15:14:53.305 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe080e35700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe07c0cbff0 0x7fe07c0affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T15:14:53.305 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.293+0000 7fe081e37700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe07c15dbc0 0x7fe07c0b04e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe07c06c8c0 tx=0x7fe074004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:53.306 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.294+0000 7fe069ffb700 1 -- 172.21.15.35:0/2902098828 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074012430 con 0x7fe07c15dbc0 2022-11-27T15:14:53.306 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.294+0000 7fe069ffb700 1 -- 172.21.15.35:0/2902098828 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe074020070 con 0x7fe07c15dbc0 2022-11-27T15:14:53.306 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.294+0000 7fe08389a700 1 -- 172.21.15.35:0/2902098828 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07c0c85a0 con 0x7fe07c15dbc0 2022-11-27T15:14:53.306 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.294+0000 7fe069ffb700 1 -- 172.21.15.35:0/2902098828 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074004280 con 0x7fe07c15dbc0 2022-11-27T15:14:53.307 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.294+0000 7fe08389a700 1 -- 172.21.15.35:0/2902098828 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c14c5a0 con 0x7fe07c15dbc0 2022-11-27T15:14:53.307 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.295+0000 7fe08389a700 1 -- 172.21.15.35:0/2902098828 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07c04e190 con 0x7fe07c15dbc0 2022-11-27T15:14:53.307 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.295+0000 7fe069ffb700 1 -- 172.21.15.35:0/2902098828 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fe07402e070 con 0x7fe07c15dbc0 2022-11-27T15:14:53.307 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.296+0000 7fe069ffb700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fe05c066590 0x7fe05c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:53.307 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.296+0000 7fe069ffb700 1 -- 172.21.15.35:0/2902098828 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe074022070 con 0x7fe07c15dbc0 2022-11-27T15:14:53.308 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.296+0000 7fe081636700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fe05c066590 0x7fe05c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:53.308 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.296+0000 7fe081636700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fe05c066590 0x7fe05c068a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe078001270 tx=0x7fe078009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:53.309 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.300+0000 7fe069ffb700 1 -- 172.21.15.35:0/2902098828 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fe074052c30 con 0x7fe07c15dbc0 2022-11-27T15:14:53.537 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.535+0000 7fe08389a700 1 -- 172.21.15.35:0/2902098828 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe07c0cd1d0 con 0x7fe07c15dbc0 2022-11-27T15:14:53.537 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.535+0000 7fe069ffb700 1 -- 172.21.15.35:0/2902098828 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7fe07402cde0 con 0x7fe07c15dbc0 2022-11-27T15:14:53.538 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:14:53.538 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.35:6835/1553785637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6833","nonce":1553785637},{"type":"v1","addr":"172.21.15.35:6835","nonce":1553785637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6837/3236550519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6835","nonce":3236550519},{"type":"v1","addr":"172.21.15.85:6837","nonce":3236550519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6836/2954315152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6834","nonce":2954315152},{"type":"v1","addr":"172.21.15.85:6836","nonce":2954315152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4299,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.35:6834/4037177741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6832","nonce":4037177741},{"type":"v1","addr":"172.21.15.35:6834","nonce":4037177741}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2022-11-27T15:14:53.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 -- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fe05c066590 msgr2=0x7fe05c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fe05c066590 0x7fe05c068a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe078001270 tx=0x7fe078009040 comp rx=0 tx=0).stop 2022-11-27T15:14:53.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 -- 172.21.15.35:0/2902098828 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe07c15dbc0 msgr2=0x7fe07c0b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe07c15dbc0 0x7fe07c0b04e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe07c06c8c0 tx=0x7fe074004ad0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.541 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 -- 172.21.15.35:0/2902098828 shutdown_connections 2022-11-27T15:14:53.541 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7fe05c066590 0x7fe05c068a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.542 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe07c15dbc0 0x7fe07c0b04e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.542 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe07c0b17b0 0x7fe07c14c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.542 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 --2- 172.21.15.35:0/2902098828 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe07c0cbff0 0x7fe07c0affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.542 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 -- 172.21.15.35:0/2902098828 >> 172.21.15.35:0/2902098828 conn(0x7fe07c06eb60 msgr2=0x7fe07c15d140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:53.542 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.537+0000 7fe08389a700 1 -- 172.21.15.35:0/2902098828 shutdown_connections 2022-11-27T15:14:53.543 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.538+0000 7fe08389a700 1 -- 172.21.15.35:0/2902098828 wait complete. 2022-11-27T15:14:53.543 INFO:teuthology.orchestra.run.smithi035.stderr:dumped fsmap epoch 3 2022-11-27T15:14:53.552 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T15:14:53.676 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.674+0000 7f7ab837a700 1 Processor -- start 2022-11-27T15:14:53.676 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.674+0000 7f7ab837a700 1 -- start start 2022-11-27T15:14:53.679 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.675+0000 7f7ab837a700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f7ab0069df0 0x7f7ab00b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:53.679 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.675+0000 7f7ab837a700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00ccb30 con 0x7f7ab006c640 2022-11-27T15:14:53.680 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.675+0000 7f7ab837a700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00b1220 con 0x7f7ab0064f70 2022-11-27T15:14:53.680 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.675+0000 7f7ab837a700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab00cc2d0 con 0x7f7ab0069df0 2022-11-27T15:14:53.680 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.675+0000 7f7ab5915700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f7ab0069df0 0x7f7ab00b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:53.680 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.675+0000 7f7ab5915700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f7ab0069df0 0x7f7ab00b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39808/0 (socket says 172.21.15.35:39808) 2022-11-27T15:14:53.680 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.675+0000 7f7ab5915700 1 -- 172.21.15.35:0/3323451996 learned_addr learned my addr 172.21.15.35:0/3323451996 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:53.681 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.676+0000 7f7ab5915700 1 -- 172.21.15.35:0/3323451996 >> v1:172.21.15.85:6789/0 conn(0x7f7ab0064f70 legacy=0x7f7ab00b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:53.681 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.676+0000 7f7ab5915700 1 -- 172.21.15.35:0/3323451996 >> v1:172.21.15.35:6789/0 conn(0x7f7ab006c640 legacy=0x7f7ab006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.681 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.676+0000 7f7ab5915700 1 -- 172.21.15.35:0/3323451996 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab006a290 con 0x7f7ab0069df0 2022-11-27T15:14:53.681 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.676+0000 7f7ab5114700 1 -- 172.21.15.35:0/3323451996 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790589660 0 0) 0x7f7ab00ccb30 con 0x7f7ab006c640 2022-11-27T15:14:53.682 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.676+0000 7f7ab5915700 1 --2- 172.21.15.35:0/3323451996 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0069df0 0x7f7ab00b16b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00098f0 tx=0x7f7aa0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af85dd6bbd02c4ce server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:53.682 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.676+0000 7f7ab5114700 1 -- 172.21.15.35:0/3323451996 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0004230 con 0x7f7ab0069df0 2022-11-27T15:14:53.682 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.676+0000 7f7ab5114700 1 -- 172.21.15.35:0/3323451996 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa0004390 con 0x7f7ab0069df0 2022-11-27T15:14:53.682 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.676+0000 7f7ab5114700 1 -- 172.21.15.35:0/3323451996 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00036a0 con 0x7f7ab0069df0 2022-11-27T15:14:53.683 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.676+0000 7f7ab837a700 1 -- 172.21.15.35:0/3323451996 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0069df0 msgr2=0x7f7ab00b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.683 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.676+0000 7f7ab837a700 1 --2- 172.21.15.35:0/3323451996 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0069df0 0x7f7ab00b16b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00098f0 tx=0x7f7aa0009c00 comp rx=0 tx=0).stop 2022-11-27T15:14:53.683 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.677+0000 7f7ab837a700 1 -- 172.21.15.35:0/3323451996 shutdown_connections 2022-11-27T15:14:53.683 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.677+0000 7f7ab837a700 1 --2- 172.21.15.35:0/3323451996 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0069df0 0x7f7ab00b16b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.683 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.677+0000 7f7ab837a700 1 -- 172.21.15.35:0/3323451996 >> 172.21.15.35:0/3323451996 conn(0x7f7ab00b0bb0 msgr2=0x7f7ab00ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:53.684 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.677+0000 7f7ab837a700 1 -- 172.21.15.35:0/3323451996 shutdown_connections 2022-11-27T15:14:53.684 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.677+0000 7f7ab837a700 1 -- 172.21.15.35:0/3323451996 wait complete. 2022-11-27T15:14:53.687 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.678+0000 7f7ab837a700 1 Processor -- start 2022-11-27T15:14:53.687 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.678+0000 7f7ab837a700 1 -- start start 2022-11-27T15:14:53.688 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.678+0000 7f7ab837a700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0064f70 0x7f7ab0154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:53.688 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.678+0000 7f7ab837a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7ab0069df0 0x7f7ab00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:53.688 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.678+0000 7f7ab837a700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ab006c640 0x7f7ab00ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:53.689 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.678+0000 7f7ab837a700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00c7e40 con 0x7f7ab0064f70 2022-11-27T15:14:53.689 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.678+0000 7f7ab837a700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab0154790 con 0x7f7ab006c640 2022-11-27T15:14:53.689 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.678+0000 7f7ab837a700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00c8f60 con 0x7f7ab0069df0 2022-11-27T15:14:53.689 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab6116700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0064f70 0x7f7ab0154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:53.690 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab6116700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0064f70 0x7f7ab0154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39812/0 (socket says 172.21.15.35:39812) 2022-11-27T15:14:53.690 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab6116700 1 -- 172.21.15.35:0/1987813378 learned_addr learned my addr 172.21.15.35:0/1987813378 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:53.690 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab6917700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ab006c640 0x7f7ab00ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:53.690 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab5915700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7ab0069df0 0x7f7ab00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:53.690 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab6116700 1 -- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7ab0069df0 msgr2=0x7f7ab00c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.691 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab6116700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7ab0069df0 0x7f7ab00c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.691 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab6116700 1 -- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ab006c640 msgr2=0x7f7ab00ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.691 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab6116700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ab006c640 0x7f7ab00ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.691 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab6116700 1 -- 172.21.15.35:0/1987813378 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa0009550 con 0x7f7ab0064f70 2022-11-27T15:14:53.692 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab5915700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7ab0069df0 0x7f7ab00c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T15:14:53.692 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab6116700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0064f70 0x7f7ab0154250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8001560 tx=0x7f7aa800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:53.692 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7a9effd700 1 -- 172.21.15.35:0/1987813378 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8003e30 con 0x7f7ab0064f70 2022-11-27T15:14:53.693 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7a9effd700 1 -- 172.21.15.35:0/1987813378 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa8005070 con 0x7f7ab0064f70 2022-11-27T15:14:53.693 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab837a700 1 -- 172.21.15.35:0/1987813378 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab00c9200 con 0x7f7ab0064f70 2022-11-27T15:14:53.693 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7a9effd700 1 -- 172.21.15.35:0/1987813378 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8005420 con 0x7f7ab0064f70 2022-11-27T15:14:53.693 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.679+0000 7f7ab837a700 1 -- 172.21.15.35:0/1987813378 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab00c97c0 con 0x7f7ab0064f70 2022-11-27T15:14:53.693 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.680+0000 7f7ab837a700 1 -- 172.21.15.35:0/1987813378 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab004e190 con 0x7f7ab0064f70 2022-11-27T15:14:53.694 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.680+0000 7f7a9effd700 1 -- 172.21.15.35:0/1987813378 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f7aa8019030 con 0x7f7ab0064f70 2022-11-27T15:14:53.694 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.681+0000 7f7a9effd700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7a900665c0 0x7f7a90068a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:53.694 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.681+0000 7f7a9effd700 1 -- 172.21.15.35:0/1987813378 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7aa8082b00 con 0x7f7ab0064f70 2022-11-27T15:14:53.694 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.681+0000 7f7ab5915700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7a900665c0 0x7f7a90068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:53.695 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.682+0000 7f7ab5915700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7a900665c0 0x7f7a90068a70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7ab01e3720 tx=0x7f7aa0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:53.695 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.685+0000 7f7a9effd700 1 -- 172.21.15.35:0/1987813378 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f7aa8050560 con 0x7f7ab0064f70 2022-11-27T15:14:53.878 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.876+0000 7f7ab837a700 1 -- 172.21.15.35:0/1987813378 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ab00b4000 con 0x7f7ab0064f70 2022-11-27T15:14:53.879 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.877+0000 7f7a9effd700 1 -- 172.21.15.35:0/1987813378 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13693 (secure 0 0 0) 0x7f7aa8017020 con 0x7f7ab0064f70 2022-11-27T15:14:53.879 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:14:53.880 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":19,"fsid":"1a1c2d0e-bda9-4965-9029-a4d82eabd38a","created":"2022-11-27T15:14:31.789352+0000","modified":"2022-11-27T15:14:53.081497+0000","last_up_change":"2022-11-27T15:14:43.038461+0000","last_in_change":"2022-11-27T15:14:34.339079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T15:14:43.200032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T15:14:50.308558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"92e0fd37-a176-44da-847e-2bc153faa513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":101387},{"type":"v1","addr":"172.21.15.35:6801","nonce":101387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":101387},{"type":"v1","addr":"172.21.15.35:6803","nonce":101387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":101387},{"type":"v1","addr":"172.21.15.35:6807","nonce":101387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":101387},{"type":"v1","addr":"172.21.15.35:6805","nonce":101387}]},"public_addr":"172.21.15.35:6801/101387","cluster_addr":"172.21.15.35:6803/101387","heartbeat_back_addr":"172.21.15.35:6807/101387","heartbeat_front_addr":"172.21.15.35:6805/101387","state":["exists","up"]},{"osd":1,"uuid":"e9828c4c-3cab-4b51-80fe-beaca70fd85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6816","nonce":101390},{"type":"v1","addr":"172.21.15.35:6817","nonce":101390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6818","nonce":101390},{"type":"v1","addr":"172.21.15.35:6819","nonce":101390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6822","nonce":101390},{"type":"v1","addr":"172.21.15.35:6823","nonce":101390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6820","nonce":101390},{"type":"v1","addr":"172.21.15.35:6821","nonce":101390}]},"public_addr":"172.21.15.35:6817/101390","cluster_addr":"172.21.15.35:6819/101390","heartbeat_back_addr":"172.21.15.35:6823/101390","heartbeat_front_addr":"172.21.15.35:6821/101390","state":["exists","up"]},{"osd":2,"uuid":"80ba487d-307a-46d3-bf27-fb037321414c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6808","nonce":101389},{"type":"v1","addr":"172.21.15.35:6809","nonce":101389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6810","nonce":101389},{"type":"v1","addr":"172.21.15.35:6811","nonce":101389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6814","nonce":101389},{"type":"v1","addr":"172.21.15.35:6815","nonce":101389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6812","nonce":101389},{"type":"v1","addr":"172.21.15.35:6813","nonce":101389}]},"public_addr":"172.21.15.35:6809/101389","cluster_addr":"172.21.15.35:6811/101389","heartbeat_back_addr":"172.21.15.35:6815/101389","heartbeat_front_addr":"172.21.15.35:6813/101389","state":["exists","up"]},{"osd":3,"uuid":"962bb8b8-e6be-48ce-99c8-4697ba5fd364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6824","nonce":101413},{"type":"v1","addr":"172.21.15.35:6825","nonce":101413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6826","nonce":101413},{"type":"v1","addr":"172.21.15.35:6827","nonce":101413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6830","nonce":101413},{"type":"v1","addr":"172.21.15.35:6831","nonce":101413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6828","nonce":101413},{"type":"v1","addr":"172.21.15.35:6829","nonce":101413}]},"public_addr":"172.21.15.35:6825/101413","cluster_addr":"172.21.15.35:6827/101413","heartbeat_back_addr":"172.21.15.35:6831/101413","heartbeat_front_addr":"172.21.15.35:6829/101413","state":["exists","up"]},{"osd":4,"uuid":"fd1b4ee8-0942-467c-b3cc-990bfb7c34c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":101623},{"type":"v1","addr":"172.21.15.85:6801","nonce":101623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":101623},{"type":"v1","addr":"172.21.15.85:6803","nonce":101623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":101623},{"type":"v1","addr":"172.21.15.85:6807","nonce":101623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":101623},{"type":"v1","addr":"172.21.15.85:6805","nonce":101623}]},"public_addr":"172.21.15.85:6801/101623","cluster_addr":"172.21.15.85:6803/101623","heartbeat_back_addr":"172.21.15.85:6807/101623","heartbeat_front_addr":"172.21.15.85:6805/101623","state":["exists","up"]},{"osd":5,"uuid":"f6133163-76e1-4748-83ad-e32cfaaaaf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":101624},{"type":"v1","addr":"172.21.15.85:6809","nonce":101624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":101624},{"type":"v1","addr":"172.21.15.85:6811","nonce":101624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":101624},{"type":"v1","addr":"172.21.15.85:6815","nonce":101624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":101624},{"type":"v1","addr":"172.21.15.85:6813","nonce":101624}]},"public_addr":"172.21.15.85:6809/101624","cluster_addr":"172.21.15.85:6811/101624","heartbeat_back_addr":"172.21.15.85:6815/101624","heartbeat_front_addr":"172.21.15.85:6813/101624","state":["exists","up"]},{"osd":6,"uuid":"9bdcaeaa-8746-4eb9-bc3c-a3a5f3269550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6816","nonce":101625},{"type":"v1","addr":"172.21.15.85:6817","nonce":101625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":101625},{"type":"v1","addr":"172.21.15.85:6819","nonce":101625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":101625},{"type":"v1","addr":"172.21.15.85:6823","nonce":101625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":101625},{"type":"v1","addr":"172.21.15.85:6821","nonce":101625}]},"public_addr":"172.21.15.85:6817/101625","cluster_addr":"172.21.15.85:6819/101625","heartbeat_back_addr":"172.21.15.85:6823/101625","heartbeat_front_addr":"172.21.15.85:6821/101625","state":["exists","up"]},{"osd":7,"uuid":"0aba1feb-9777-4a35-892d-f1ff9fec1d80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":101668},{"type":"v1","addr":"172.21.15.85:6825","nonce":101668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":101668},{"type":"v1","addr":"172.21.15.85:6827","nonce":101668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":101668},{"type":"v1","addr":"172.21.15.85:6831","nonce":101668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":101668},{"type":"v1","addr":"172.21.15.85:6829","nonce":101668}]},"public_addr":"172.21.15.85:6825/101668","cluster_addr":"172.21.15.85:6827/101668","heartbeat_back_addr":"172.21.15.85:6831/101668","heartbeat_front_addr":"172.21.15.85:6829/101668","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.290815+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.534039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.488830+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.508610+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.233117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.549218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.502641+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:14:39.594384+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T15:14:53.881 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 -- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7a900665c0 msgr2=0x7f7a90068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.881 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7a900665c0 0x7f7a90068a70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7ab01e3720 tx=0x7f7aa0008030 comp rx=0 tx=0).stop 2022-11-27T15:14:53.882 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 -- 172.21.15.35:0/1987813378 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0064f70 msgr2=0x7f7ab0154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:53.882 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0064f70 0x7f7ab0154250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8001560 tx=0x7f7aa800ce50 comp rx=0 tx=0).stop 2022-11-27T15:14:53.882 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 -- 172.21.15.35:0/1987813378 shutdown_connections 2022-11-27T15:14:53.882 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7a900665c0 0x7f7a90068a70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.883 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ab006c640 0x7f7ab00ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.883 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ab0064f70 0x7f7ab0154250 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.883 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 --2- 172.21.15.35:0/1987813378 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7ab0069df0 0x7f7ab00c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:53.883 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 -- 172.21.15.35:0/1987813378 >> 172.21.15.35:0/1987813378 conn(0x7f7ab00b0bb0 msgr2=0x7f7ab00b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:53.884 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 -- 172.21.15.35:0/1987813378 shutdown_connections 2022-11-27T15:14:53.884 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:53.879+0000 7f7ab837a700 1 -- 172.21.15.35:0/1987813378 wait complete. 2022-11-27T15:14:53.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-11-27T15:14:43.200032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T15:14:50.308558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T15:14:53.894 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T15:14:54.021 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.019+0000 7f0cf1131700 1 Processor -- start 2022-11-27T15:14:54.021 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.019+0000 7f0cf1131700 1 -- start start 2022-11-27T15:14:54.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cf1131700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f0cec0cc670 0x7f0cec0c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cf1131700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f0cec06aaf0 0x7f0cec06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cf1131700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cec070780 con 0x7f0cec15b5d0 2022-11-27T15:14:54.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cf1131700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cec070a60 con 0x7f0cec0cc670 2022-11-27T15:14:54.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cf1131700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cec070d40 con 0x7f0cec06aaf0 2022-11-27T15:14:54.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cea59c700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f0cec0cc670 0x7f0cec0c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.026 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cea59c700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f0cec0cc670 0x7f0cec0c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39822/0 (socket says 172.21.15.35:39822) 2022-11-27T15:14:54.026 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cea59c700 1 -- 172.21.15.35:0/2631477197 learned_addr learned my addr 172.21.15.35:0/2631477197 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:54.026 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0ceb59e700 1 --2- 172.21.15.35:0/2631477197 >> v2:172.21.15.85:3300/0 conn(0x7f0cec06aaf0 0x7f0cec06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.026 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cea59c700 1 -- 172.21.15.35:0/2631477197 >> v2:172.21.15.85:3300/0 conn(0x7f0cec06aaf0 msgr2=0x7f0cec06dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:54.027 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cea59c700 1 --2- 172.21.15.35:0/2631477197 >> v2:172.21.15.85:3300/0 conn(0x7f0cec06aaf0 0x7f0cec06dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.027 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cea59c700 1 -- 172.21.15.35:0/2631477197 >> v1:172.21.15.85:6789/0 conn(0x7f0cec15b5d0 legacy=0x7f0cec0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:54.027 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cea59c700 1 -- 172.21.15.35:0/2631477197 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec071020 con 0x7f0cec0cc670 2022-11-27T15:14:54.027 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.020+0000 7f0cea59c700 1 --2- 172.21.15.35:0/2631477197 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 0x7f0cec0c89e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0cd660 tx=0x7f0cdc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12303b95706f0e06 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0ce9d9b700 1 -- 172.21.15.35:0/2631477197 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc004230 con 0x7f0cec0cc670 2022-11-27T15:14:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0ce9d9b700 1 -- 172.21.15.35:0/2631477197 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc004390 con 0x7f0cec0cc670 2022-11-27T15:14:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0ce9d9b700 1 -- 172.21.15.35:0/2631477197 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0036a0 con 0x7f0cec0cc670 2022-11-27T15:14:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0cf1131700 1 -- 172.21.15.35:0/2631477197 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 msgr2=0x7f0cec0c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.029 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0cf1131700 1 --2- 172.21.15.35:0/2631477197 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 0x7f0cec0c89e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0cd660 tx=0x7f0cdc009bd0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.029 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0cf1131700 1 -- 172.21.15.35:0/2631477197 shutdown_connections 2022-11-27T15:14:54.029 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0cf1131700 1 --2- 172.21.15.35:0/2631477197 >> v2:172.21.15.85:3300/0 conn(0x7f0cec06aaf0 0x7f0cec06dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.029 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0cf1131700 1 --2- 172.21.15.35:0/2631477197 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 0x7f0cec0c89e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.030 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0cf1131700 1 -- 172.21.15.35:0/2631477197 >> 172.21.15.35:0/2631477197 conn(0x7f0cec0aff10 msgr2=0x7f0cec0b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:54.030 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0cf1131700 1 -- 172.21.15.35:0/2631477197 shutdown_connections 2022-11-27T15:14:54.031 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.021+0000 7f0cf1131700 1 -- 172.21.15.35:0/2631477197 wait complete. 2022-11-27T15:14:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.023+0000 7f0cf1131700 1 Processor -- start 2022-11-27T15:14:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.023+0000 7f0cf1131700 1 -- start start 2022-11-27T15:14:54.035 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0cf1131700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0cec06aaf0 0x7f0cec0c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.035 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0cf1131700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 0x7f0cec0c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.035 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0cf1131700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0cec15b5d0 0x7f0cec1f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.036 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0cf1131700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec14b2b0 con 0x7f0cec0cc670 2022-11-27T15:14:54.036 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0cf1131700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec1f1b20 con 0x7f0cec06aaf0 2022-11-27T15:14:54.036 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0cf1131700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec14ac60 con 0x7f0cec15b5d0 2022-11-27T15:14:54.037 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0cea59c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 0x7f0cec0c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.037 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0cea59c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 0x7f0cec0c45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39828/0 (socket says 172.21.15.35:39828) 2022-11-27T15:14:54.037 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0cea59c700 1 -- 172.21.15.35:0/2717955460 learned_addr learned my addr 172.21.15.35:0/2717955460 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:54.037 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0cead9d700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0cec06aaf0 0x7f0cec0c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.038 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.024+0000 7f0ceb59e700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0cec15b5d0 0x7f0cec1f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.038 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cea59c700 1 -- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0cec15b5d0 msgr2=0x7f0cec1f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.038 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cea59c700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0cec15b5d0 0x7f0cec1f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.038 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cea59c700 1 -- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0cec06aaf0 msgr2=0x7f0cec0c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.039 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cea59c700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0cec06aaf0 0x7f0cec0c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.039 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cea59c700 1 -- 172.21.15.35:0/2717955460 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc009550 con 0x7f0cec0cc670 2022-11-27T15:14:54.039 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cea59c700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 0x7f0cec0c45b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc004000 tx=0x7f0cdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:54.039 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cd37fe700 1 -- 172.21.15.35:0/2717955460 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc003c90 con 0x7f0cec0cc670 2022-11-27T15:14:54.040 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cd37fe700 1 -- 172.21.15.35:0/2717955460 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc01f070 con 0x7f0cec0cc670 2022-11-27T15:14:54.040 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cf1131700 1 -- 172.21.15.35:0/2717955460 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cec14af40 con 0x7f0cec0cc670 2022-11-27T15:14:54.040 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cd37fe700 1 -- 172.21.15.35:0/2717955460 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc011870 con 0x7f0cec0cc670 2022-11-27T15:14:54.041 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.025+0000 7f0cf1131700 1 -- 172.21.15.35:0/2717955460 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cec14b920 con 0x7f0cec0cc670 2022-11-27T15:14:54.041 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.026+0000 7f0cf1131700 1 -- 172.21.15.35:0/2717955460 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cec14aac0 con 0x7f0cec0cc670 2022-11-27T15:14:54.041 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.026+0000 7f0cd37fe700 1 -- 172.21.15.35:0/2717955460 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f0cdc017020 con 0x7f0cec0cc670 2022-11-27T15:14:54.041 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.027+0000 7f0cd37fe700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0ccc066590 0x7f0ccc068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.042 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.027+0000 7f0cd37fe700 1 -- 172.21.15.35:0/2717955460 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0cdc084b30 con 0x7f0cec0cc670 2022-11-27T15:14:54.042 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.027+0000 7f0cead9d700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0ccc066590 0x7f0ccc068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.042 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.027+0000 7f0cead9d700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0ccc066590 0x7f0ccc068a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0c8920 tx=0x7f0ce00090e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:54.042 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.031+0000 7f0cd37fe700 1 -- 172.21.15.35:0/2717955460 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f0cec14aac0 con 0x7f0cec0cc670 2022-11-27T15:14:54.224 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.222+0000 7f0cf1131700 1 -- 172.21.15.35:0/2717955460 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0cec14aac0 con 0x7f0cec0cc670 2022-11-27T15:14:54.224 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.222+0000 7f0cd37fe700 1 -- 172.21.15.35:0/2717955460 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0cec14aac0 con 0x7f0cec0cc670 2022-11-27T15:14:54.224 INFO:teuthology.orchestra.run.smithi035.stdout:pg_num: 1 2022-11-27T15:14:54.226 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 -- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0ccc066590 msgr2=0x7f0ccc068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.226 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0ccc066590 0x7f0ccc068a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0c8920 tx=0x7f0ce00090e0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.227 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 -- 172.21.15.35:0/2717955460 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 msgr2=0x7f0cec0c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.227 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 0x7f0cec0c45b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc004000 tx=0x7f0cdc004750 comp rx=0 tx=0).stop 2022-11-27T15:14:54.227 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 -- 172.21.15.35:0/2717955460 shutdown_connections 2022-11-27T15:14:54.227 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f0ccc066590 0x7f0ccc068a40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.228 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0cec15b5d0 0x7f0cec1f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.228 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0cec06aaf0 0x7f0cec0c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.228 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 --2- 172.21.15.35:0/2717955460 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f0cec0cc670 0x7f0cec0c45b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.228 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 -- 172.21.15.35:0/2717955460 >> 172.21.15.35:0/2717955460 conn(0x7f0cec0aff10 msgr2=0x7f0cec06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:54.228 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 -- 172.21.15.35:0/2717955460 shutdown_connections 2022-11-27T15:14:54.229 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.224+0000 7f0cf1131700 1 -- 172.21.15.35:0/2717955460 wait complete. 2022-11-27T15:14:54.238 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T15:14:54.365 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.364+0000 7f9ad9343700 1 Processor -- start 2022-11-27T15:14:54.366 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.364+0000 7f9ad9343700 1 -- start start 2022-11-27T15:14:54.373 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.364+0000 7f9ad9343700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f9ad40d01d0 0x7f9ad40ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.373 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.364+0000 7f9ad9343700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad40ccda0 con 0x7f9ad40d2740 2022-11-27T15:14:54.374 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.364+0000 7f9ad9343700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad40d1cf0 con 0x7f9ad4159390 2022-11-27T15:14:54.374 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.364+0000 7f9ad9343700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad40d0650 con 0x7f9ad40d01d0 2022-11-27T15:14:54.374 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad27fc700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f9ad40d2740 0x7f9ad40cfac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:51232/0 (socket says 172.21.15.35:51232) 2022-11-27T15:14:54.374 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad27fc700 1 -- 172.21.15.35:0/3122524814 learned_addr learned my addr 172.21.15.35:0/3122524814 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-27T15:14:54.375 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621088351 0 0) 0x7f9ad40ccda0 con 0x7f9ad40d2740 2022-11-27T15:14:54.375 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad37fe700 1 --2- 172.21.15.35:0/3122524814 >> v2:172.21.15.85:3300/0 conn(0x7f9ad40d01d0 0x7f9ad40ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.375 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab40035b0 con 0x7f9ad40d2740 2022-11-27T15:14:54.375 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 <== mon.0 v1:172.21.15.35:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 918929229 0 0) 0x7f9ab40035b0 con 0x7f9ad40d2740 2022-11-27T15:14:54.376 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 >> v2:172.21.15.85:3300/0 conn(0x7f9ad40d01d0 msgr2=0x7f9ad40ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:54.376 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 --2- 172.21.15.35:0/3122524814 >> v2:172.21.15.85:3300/0 conn(0x7f9ad40d01d0 0x7f9ad40ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.376 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 >> v1:172.21.15.85:6789/0 conn(0x7f9ad4159390 legacy=0x7f9ad415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.376 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 --> v1:172.21.15.35:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad40d22b0 con 0x7f9ad40d2740 2022-11-27T15:14:54.376 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 <== mon.0 v1:172.21.15.35:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1516710402 0 0) 0x7f9abc001670 con 0x7f9ad40d2740 2022-11-27T15:14:54.377 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 >> v1:172.21.15.35:6789/0 conn(0x7f9ad40d2740 legacy=0x7f9ad40cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.377 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9ab4003a00 0x7f9ab4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.377 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ab4004850 0x7f9ab4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.377 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9ab4005200 0x7f9ab400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.378 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40d0650 con 0x7f9ab4004850 2022-11-27T15:14:54.378 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab400c080 con 0x7f9ab4003a00 2022-11-27T15:14:54.378 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab400c240 con 0x7f9ab4005200 2022-11-27T15:14:54.378 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad27fc700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ab4004850 0x7f9ab4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.379 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad2ffd700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9ab4003a00 0x7f9ab4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.379 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.365+0000 7f9ad37fe700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9ab4005200 0x7f9ab400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.379 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad27fc700 1 -- 172.21.15.35:0/3122524814 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9ab4005200 msgr2=0x7f9ab400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:54.379 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad27fc700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9ab4005200 0x7f9ab400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.379 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad27fc700 1 -- 172.21.15.35:0/3122524814 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9ab4003a00 msgr2=0x7f9ab4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:54.380 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad27fc700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9ab4003a00 0x7f9ab4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.380 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad37fe700 1 -- 172.21.15.35:0/3122524814 reap_dead start 2022-11-27T15:14:54.380 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad27fc700 1 -- 172.21.15.35:0/3122524814 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40035b0 con 0x7f9ab4004850 2022-11-27T15:14:54.380 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad27fc700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ab4004850 0x7f9ab4004cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c240 tx=0x7f9ab4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ae099053340d29 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:54.381 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc007c10 con 0x7f9ab4004850 2022-11-27T15:14:54.381 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc00cd60 con 0x7f9ab4004850 2022-11-27T15:14:54.381 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad1ffb700 1 -- 172.21.15.35:0/3122524814 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc005ba0 con 0x7f9ab4004850 2022-11-27T15:14:54.381 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad9343700 1 -- 172.21.15.35:0/3122524814 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ab4004850 msgr2=0x7f9ab4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.382 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad9343700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ab4004850 0x7f9ab4004cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c240 tx=0x7f9ab4005880 comp rx=0 tx=0).stop 2022-11-27T15:14:54.382 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad9343700 1 -- 172.21.15.35:0/3122524814 shutdown_connections 2022-11-27T15:14:54.382 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad9343700 1 --2- 172.21.15.35:0/3122524814 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ab4004850 0x7f9ab4004cc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.382 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad9343700 1 -- 172.21.15.35:0/3122524814 >> 172.21.15.35:0/3122524814 conn(0x7f9ad4070b40 msgr2=0x7f9ad4070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:54.383 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.366+0000 7f9ad9343700 1 -- 172.21.15.35:0/3122524814 shutdown_connections 2022-11-27T15:14:54.383 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.367+0000 7f9ad9343700 1 -- 172.21.15.35:0/3122524814 wait complete. 2022-11-27T15:14:54.383 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.368+0000 7f9ad9343700 1 Processor -- start 2022-11-27T15:14:54.383 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.368+0000 7f9ad9343700 1 -- start start 2022-11-27T15:14:54.383 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad9343700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9ab4004850 0x7f9ad41f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.384 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad9343700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9ad414a310 0x7f9ad40b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.384 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad9343700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ad40b74b0 0x7f9ad40cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.384 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad9343700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40b79d0 con 0x7f9ad40b74b0 2022-11-27T15:14:54.384 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad9343700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40b7cb0 con 0x7f9ad414a310 2022-11-27T15:14:54.385 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad9343700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad41f4500 con 0x7f9ab4004850 2022-11-27T15:14:54.385 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ad40b74b0 0x7f9ad40cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.385 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ad40b74b0 0x7f9ad40cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39850/0 (socket says 172.21.15.35:39850) 2022-11-27T15:14:54.385 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad37fe700 1 -- 172.21.15.35:0/1215436434 learned_addr learned my addr 172.21.15.35:0/1215436434 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:54.386 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad27fc700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9ad414a310 0x7f9ad40b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.386 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad2ffd700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9ab4004850 0x7f9ad41f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.386 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad37fe700 1 -- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9ab4004850 msgr2=0x7f9ad41f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.386 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad37fe700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9ab4004850 0x7f9ad41f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.386 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad37fe700 1 -- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9ad414a310 msgr2=0x7f9ad40b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.387 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad37fe700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9ad414a310 0x7f9ad40b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.387 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad37fe700 1 -- 172.21.15.35:0/1215436434 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad41f47e0 con 0x7f9ad40b74b0 2022-11-27T15:14:54.387 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad37fe700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ad40b74b0 0x7f9ad40cf4b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40cdf20 tx=0x7f9acc011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:54.387 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9acb7fe700 1 -- 172.21.15.35:0/1215436434 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00e040 con 0x7f9ad40b74b0 2022-11-27T15:14:54.388 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9acb7fe700 1 -- 172.21.15.35:0/1215436434 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc00f040 con 0x7f9ad40b74b0 2022-11-27T15:14:54.388 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9acb7fe700 1 -- 172.21.15.35:0/1215436434 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00fdb0 con 0x7f9ad40b74b0 2022-11-27T15:14:54.388 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad9343700 1 -- 172.21.15.35:0/1215436434 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad41f4ac0 con 0x7f9ad40b74b0 2022-11-27T15:14:54.388 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.369+0000 7f9ad9343700 1 -- 172.21.15.35:0/1215436434 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad414d890 con 0x7f9ad40b74b0 2022-11-27T15:14:54.389 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.370+0000 7f9ad9343700 1 -- 172.21.15.35:0/1215436434 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad40cd550 con 0x7f9ad40b74b0 2022-11-27T15:14:54.389 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.371+0000 7f9acb7fe700 1 -- 172.21.15.35:0/1215436434 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f9acc022460 con 0x7f9ad40b74b0 2022-11-27T15:14:54.389 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.371+0000 7f9acb7fe700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9ab80665c0 0x7f9ab8068a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.389 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.371+0000 7f9acb7fe700 1 -- 172.21.15.35:0/1215436434 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9acc083b80 con 0x7f9ad40b74b0 2022-11-27T15:14:54.390 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.371+0000 7f9ad2ffd700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9ab80665c0 0x7f9ab8068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.390 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.372+0000 7f9ad2ffd700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9ab80665c0 0x7f9ab8068a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4007fd0 tx=0x7f9ac4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:54.390 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.375+0000 7f9acb7fe700 1 -- 172.21.15.35:0/1215436434 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f9acc051580 con 0x7f9ad40b74b0 2022-11-27T15:14:54.568 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.566+0000 7f9ad9343700 1 -- 172.21.15.35:0/1215436434 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9ad404e190 con 0x7f9ad40b74b0 2022-11-27T15:14:54.568 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.566+0000 7f9acb7fe700 1 -- 172.21.15.35:0/1215436434 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9acc054ba0 con 0x7f9ad40b74b0 2022-11-27T15:14:54.568 INFO:teuthology.orchestra.run.smithi035.stdout:pg_num: 8 2022-11-27T15:14:54.570 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 -- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9ab80665c0 msgr2=0x7f9ab8068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.570 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9ab80665c0 0x7f9ab8068a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4007fd0 tx=0x7f9ac4009040 comp rx=0 tx=0).stop 2022-11-27T15:14:54.571 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 -- 172.21.15.35:0/1215436434 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ad40b74b0 msgr2=0x7f9ad40cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.571 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ad40b74b0 0x7f9ad40cf4b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40cdf20 tx=0x7f9acc011f80 comp rx=0 tx=0).stop 2022-11-27T15:14:54.571 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 -- 172.21.15.35:0/1215436434 shutdown_connections 2022-11-27T15:14:54.571 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9ab80665c0 0x7f9ab8068a70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9ad40b74b0 0x7f9ad40cf4b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9ab4004850 0x7f9ad41f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 --2- 172.21.15.35:0/1215436434 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9ad414a310 0x7f9ad40b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 -- 172.21.15.35:0/1215436434 >> 172.21.15.35:0/1215436434 conn(0x7f9ad4070b40 msgr2=0x7f9ad415d020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:54.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 -- 172.21.15.35:0/1215436434 shutdown_connections 2022-11-27T15:14:54.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.568+0000 7f9ad9343700 1 -- 172.21.15.35:0/1215436434 wait complete. 2022-11-27T15:14:54.582 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-11-27T15:14:54.706 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.704+0000 7f7ee0bd6700 1 Processor -- start 2022-11-27T15:14:54.707 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.704+0000 7f7ee0bd6700 1 -- start start 2022-11-27T15:14:54.714 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.705+0000 7f7ee0bd6700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f7edc15b940 0x7f7edc160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.715 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.705+0000 7f7ee0bd6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc06cd50 0x7f7edc0709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.715 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.705+0000 7f7ee0bd6700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc15fb80 con 0x7f7edc070570 2022-11-27T15:14:54.715 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.705+0000 7f7ee0bd6700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7edc070fc0 con 0x7f7edc15b940 2022-11-27T15:14:54.716 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.705+0000 7f7ee0bd6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0712a0 con 0x7f7edc06cd50 2022-11-27T15:14:54.716 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.705+0000 7f7ed9d9b700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f7edc070570 0x7f7edc06c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:51252/0 (socket says 172.21.15.35:51252) 2022-11-27T15:14:54.716 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.705+0000 7f7ed9d9b700 1 -- 172.21.15.35:0/1174599186 learned_addr learned my addr 172.21.15.35:0/1174599186 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-27T15:14:54.716 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.705+0000 7f7eda59c700 1 --2- 172.21.15.35:0/1174599186 >> v2:172.21.15.85:3300/0 conn(0x7f7edc15b940 0x7f7edc160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.716 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.705+0000 7f7edad9d700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc06cd50 0x7f7edc0709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.717 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708083811 0 0) 0x7f7edc15fb80 con 0x7f7edc070570 2022-11-27T15:14:54.717 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec00035b0 con 0x7f7edc070570 2022-11-27T15:14:54.717 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 <== mon.0 v1:172.21.15.35:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2906172497 0 0) 0x7f7ec00035b0 con 0x7f7edc070570 2022-11-27T15:14:54.717 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc06cd50 msgr2=0x7f7edc0709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:54.718 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc06cd50 0x7f7edc0709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.718 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 >> v2:172.21.15.85:3300/0 conn(0x7f7edc15b940 msgr2=0x7f7edc160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:54.718 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 --2- 172.21.15.35:0/1174599186 >> v2:172.21.15.85:3300/0 conn(0x7f7edc15b940 0x7f7edc160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.718 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 --> v1:172.21.15.35:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc15f530 con 0x7f7edc070570 2022-11-27T15:14:54.718 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 <== mon.0 v1:172.21.15.35:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1516710402 0 0) 0x7f7ecc001670 con 0x7f7edc070570 2022-11-27T15:14:54.719 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 >> v1:172.21.15.35:6789/0 conn(0x7f7edc070570 legacy=0x7f7edc06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.719 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7ec0003a70 0x7f7ec0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.719 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ec0004850 0x7f7ec0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.720 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ec0005200 0x7f7ec000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.720 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc070fc0 con 0x7f7ec0004850 2022-11-27T15:14:54.720 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0712a0 con 0x7f7ec0005200 2022-11-27T15:14:54.720 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec000bf50 con 0x7f7ec0003a70 2022-11-27T15:14:54.721 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7edad9d700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc06cd50 0x7f7edc0709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:54.721 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7eda59c700 1 --2- 172.21.15.35:0/1174599186 >> v2:172.21.15.85:3300/0 conn(0x7f7edc15b940 0x7f7edc160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:14:54.721 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed9d9b700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ec0004850 0x7f7ec0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.721 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed9d9b700 1 -- 172.21.15.35:0/1174599186 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7ec0003a70 msgr2=0x7f7ec0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:54.721 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed9d9b700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7ec0003a70 0x7f7ec0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.722 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed9d9b700 1 -- 172.21.15.35:0/1174599186 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ec0005200 msgr2=0x7f7ec000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:54.722 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed9d9b700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ec0005200 0x7f7ec000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.722 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed9d9b700 1 -- 172.21.15.35:0/1174599186 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00035b0 con 0x7f7ec0004850 2022-11-27T15:14:54.722 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.706+0000 7f7ed9d9b700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ec0004850 0x7f7ec0004cc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c350 tx=0x7f7ec0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2361806d1ee78e19 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:54.723 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.707+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00cd70 con 0x7f7ec0004850 2022-11-27T15:14:54.723 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.707+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ecc005960 con 0x7f7ec0004850 2022-11-27T15:14:54.723 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.707+0000 7f7ed959a700 1 -- 172.21.15.35:0/1174599186 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc005d10 con 0x7f7ec0004850 2022-11-27T15:14:54.723 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.707+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/1174599186 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ec0004850 msgr2=0x7f7ec0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.724 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.707+0000 7f7ee0bd6700 1 --2- 172.21.15.35:0/1174599186 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7ec0004850 0x7f7ec0004cc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c350 tx=0x7f7ec0005880 comp rx=0 tx=0).stop 2022-11-27T15:14:54.724 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.707+0000 7f7edad9d700 1 -- 172.21.15.35:0/1174599186 reap_dead start 2022-11-27T15:14:54.724 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.707+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/1174599186 shutdown_connections 2022-11-27T15:14:54.724 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.707+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/1174599186 >> 172.21.15.35:0/1174599186 conn(0x7f7edc0b0bb0 msgr2=0x7f7edc0ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:54.724 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.707+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/1174599186 shutdown_connections 2022-11-27T15:14:54.725 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.707+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/1174599186 wait complete. 2022-11-27T15:14:54.725 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7ee0bd6700 1 Processor -- start 2022-11-27T15:14:54.725 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7ee0bd6700 1 -- start start 2022-11-27T15:14:54.725 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7ee0bd6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ec0004850 0x7f7edc1564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.726 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7ee0bd6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc156ba0 0x7f7edc15c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.726 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7ee0bd6700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7edc0ca330 0x7f7edc15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.726 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7ee0bd6700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0c36b0 con 0x7f7edc0ca330 2022-11-27T15:14:54.726 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7ee0bd6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc1569e0 con 0x7f7ec0004850 2022-11-27T15:14:54.726 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7ee0bd6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc15d020 con 0x7f7edc156ba0 2022-11-27T15:14:54.727 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7edad9d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7edc0ca330 0x7f7edc15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.727 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7edad9d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7edc0ca330 0x7f7edc15df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39870/0 (socket says 172.21.15.35:39870) 2022-11-27T15:14:54.727 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.709+0000 7f7edad9d700 1 -- 172.21.15.35:0/3360134542 learned_addr learned my addr 172.21.15.35:0/3360134542 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:54.727 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7eda59c700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ec0004850 0x7f7edc1564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.728 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7ed9d9b700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc156ba0 0x7f7edc15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.728 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7edad9d700 1 -- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc156ba0 msgr2=0x7f7edc15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.728 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7edad9d700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc156ba0 0x7f7edc15c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.728 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7edad9d700 1 -- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ec0004850 msgr2=0x7f7edc1564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.729 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7edad9d700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ec0004850 0x7f7edc1564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.729 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7edad9d700 1 -- 172.21.15.35:0/3360134542 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc15d300 con 0x7f7edc0ca330 2022-11-27T15:14:54.729 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7ed9d9b700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc156ba0 0x7f7edc15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T15:14:54.729 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7edad9d700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7edc0ca330 0x7f7edc15df50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7edc070520 tx=0x7f7ed400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:54.729 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7ecaffd700 1 -- 172.21.15.35:0/3360134542 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed400cc90 con 0x7f7edc0ca330 2022-11-27T15:14:54.730 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7ecaffd700 1 -- 172.21.15.35:0/3360134542 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed400f040 con 0x7f7edc0ca330 2022-11-27T15:14:54.730 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7ecaffd700 1 -- 172.21.15.35:0/3360134542 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4003a20 con 0x7f7edc0ca330 2022-11-27T15:14:54.730 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/3360134542 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc0c2ff0 con 0x7f7edc0ca330 2022-11-27T15:14:54.730 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.710+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/3360134542 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc15cc60 con 0x7f7edc0ca330 2022-11-27T15:14:54.731 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.711+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/3360134542 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc0c37f0 con 0x7f7edc0ca330 2022-11-27T15:14:54.731 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.711+0000 7f7ecaffd700 1 -- 172.21.15.35:0/3360134542 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f7ed401d030 con 0x7f7edc0ca330 2022-11-27T15:14:54.731 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.712+0000 7f7ecaffd700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7eb8066590 0x7f7eb8068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:54.731 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.712+0000 7f7ecaffd700 1 -- 172.21.15.35:0/3360134542 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7ed4083150 con 0x7f7edc0ca330 2022-11-27T15:14:54.732 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.712+0000 7f7eda59c700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7eb8066590 0x7f7eb8068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:54.732 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.712+0000 7f7eda59c700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7eb8066590 0x7f7eb8068a40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0c5260 tx=0x7f7ed0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:54.732 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.716+0000 7f7ecaffd700 1 -- 172.21.15.35:0/3360134542 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f7edc0c37f0 con 0x7f7edc0ca330 2022-11-27T15:14:54.911 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.909+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/3360134542 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7edc0c37f0 con 0x7f7edc0ca330 2022-11-27T15:14:54.911 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.909+0000 7f7ecaffd700 1 -- 172.21.15.35:0/3360134542 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7ed401b090 con 0x7f7edc0ca330 2022-11-27T15:14:54.912 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-27T15:14:54.912 INFO:teuthology.orchestra.run.smithi035.stdout:[ 2022-11-27T15:14:54.913 INFO:teuthology.orchestra.run.smithi035.stdout: { 2022-11-27T15:14:54.913 INFO:teuthology.orchestra.run.smithi035.stdout: "poolnum": 1, 2022-11-27T15:14:54.913 INFO:teuthology.orchestra.run.smithi035.stdout: "poolname": ".mgr" 2022-11-27T15:14:54.914 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2022-11-27T15:14:54.914 INFO:teuthology.orchestra.run.smithi035.stdout: { 2022-11-27T15:14:54.915 INFO:teuthology.orchestra.run.smithi035.stdout: "poolnum": 2, 2022-11-27T15:14:54.915 INFO:teuthology.orchestra.run.smithi035.stdout: "poolname": "rbd" 2022-11-27T15:14:54.915 INFO:teuthology.orchestra.run.smithi035.stdout: } 2022-11-27T15:14:54.915 INFO:teuthology.orchestra.run.smithi035.stdout:] 2022-11-27T15:14:54.917 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7eb8066590 msgr2=0x7f7eb8068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.917 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7eb8066590 0x7f7eb8068a40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0c5260 tx=0x7f7ed0008040 comp rx=0 tx=0).stop 2022-11-27T15:14:54.917 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/3360134542 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7edc0ca330 msgr2=0x7f7edc15df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:54.918 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7edc0ca330 0x7f7edc15df50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7edc070520 tx=0x7f7ed400c3a0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.918 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/3360134542 shutdown_connections 2022-11-27T15:14:54.918 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f7eb8066590 0x7f7eb8068a40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.918 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7edc0ca330 0x7f7edc15df50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.919 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7ec0004850 0x7f7edc1564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.919 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 --2- 172.21.15.35:0/3360134542 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7edc156ba0 0x7f7edc15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:54.919 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/3360134542 >> 172.21.15.35:0/3360134542 conn(0x7f7edc0b0bb0 msgr2=0x7f7edc06c640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:54.920 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/3360134542 shutdown_connections 2022-11-27T15:14:54.920 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:54.911+0000 7f7ee0bd6700 1 -- 172.21.15.35:0/3360134542 wait complete. 2022-11-27T15:14:54.925 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-11-27T15:14:54.926 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2022-11-27T15:14:55.054 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.052+0000 7f98a98e6700 1 Processor -- start 2022-11-27T15:14:55.055 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.053+0000 7f98a98e6700 1 -- start start 2022-11-27T15:14:55.058 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.053+0000 7f98a98e6700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f98a40ce6d0 0x7f98a40d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:55.058 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.053+0000 7f98a98e6700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f98a40ceb50 0x7f98a40cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:55.058 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.053+0000 7f98a98e6700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a40cc7c0 con 0x7f98a415c470 2022-11-27T15:14:55.058 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.053+0000 7f98a98e6700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a415cf50 con 0x7f98a40ceb50 2022-11-27T15:14:55.059 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.053+0000 7f98a98e6700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a4152ec0 con 0x7f98a40ce6d0 2022-11-27T15:14:55.059 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.053+0000 7f98a37fe700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f98a40ceb50 0x7f98a40cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:55.059 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a37fe700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f98a40ceb50 0x7f98a40cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39878/0 (socket says 172.21.15.35:39878) 2022-11-27T15:14:55.059 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a37fe700 1 -- 172.21.15.35:0/846066113 learned_addr learned my addr 172.21.15.35:0/846066113 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:55.059 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a27fc700 1 --2- 172.21.15.35:0/846066113 >> v2:172.21.15.85:3300/0 conn(0x7f98a40ce6d0 0x7f98a40d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:55.060 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a37fe700 1 -- 172.21.15.35:0/846066113 >> v2:172.21.15.85:3300/0 conn(0x7f98a40ce6d0 msgr2=0x7f98a40d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:55.060 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a37fe700 1 --2- 172.21.15.35:0/846066113 >> v2:172.21.15.85:3300/0 conn(0x7f98a40ce6d0 0x7f98a40d28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:55.060 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a37fe700 1 -- 172.21.15.35:0/846066113 >> v1:172.21.15.85:6789/0 conn(0x7f98a415c470 legacy=0x7f98a415c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:55.060 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a37fe700 1 -- 172.21.15.35:0/846066113 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a40cf490 con 0x7f98a40ceb50 2022-11-27T15:14:55.061 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a37fe700 1 --2- 172.21.15.35:0/846066113 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a40ceb50 0x7f98a40cef50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f98a408b0c0 tx=0x7f989c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc167b2ddfbf43c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:55.061 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a1ffb700 1 -- 172.21.15.35:0/846066113 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00e070 con 0x7f98a40ceb50 2022-11-27T15:14:55.061 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a1ffb700 1 -- 172.21.15.35:0/846066113 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c00b9c0 con 0x7f98a40ceb50 2022-11-27T15:14:55.061 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.054+0000 7f98a1ffb700 1 -- 172.21.15.35:0/846066113 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00bd70 con 0x7f98a40ceb50 2022-11-27T15:14:55.062 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.055+0000 7f98a98e6700 1 -- 172.21.15.35:0/846066113 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a40ceb50 msgr2=0x7f98a40cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:55.062 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.055+0000 7f98a98e6700 1 --2- 172.21.15.35:0/846066113 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a40ceb50 0x7f98a40cef50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f98a408b0c0 tx=0x7f989c00b330 comp rx=0 tx=0).stop 2022-11-27T15:14:55.062 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.055+0000 7f98a98e6700 1 -- 172.21.15.35:0/846066113 shutdown_connections 2022-11-27T15:14:55.062 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.055+0000 7f98a98e6700 1 --2- 172.21.15.35:0/846066113 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a40ceb50 0x7f98a40cef50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:55.063 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.055+0000 7f98a98e6700 1 --2- 172.21.15.35:0/846066113 >> v2:172.21.15.85:3300/0 conn(0x7f98a40ce6d0 0x7f98a40d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:55.063 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.055+0000 7f98a98e6700 1 -- 172.21.15.35:0/846066113 >> 172.21.15.35:0/846066113 conn(0x7f98a4070b40 msgr2=0x7f98a4070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:55.063 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.055+0000 7f98a98e6700 1 -- 172.21.15.35:0/846066113 shutdown_connections 2022-11-27T15:14:55.063 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.055+0000 7f98a98e6700 1 -- 172.21.15.35:0/846066113 wait complete. 2022-11-27T15:14:55.064 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.056+0000 7f98a98e6700 1 Processor -- start 2022-11-27T15:14:55.064 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.056+0000 7f98a98e6700 1 -- start start 2022-11-27T15:14:55.067 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a98e6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98a40ce6d0 0x7f98a4151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:55.067 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a98e6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98a40ceb50 0x7f98a41f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:55.067 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a98e6700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a415c470 0x7f98a41f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:55.068 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a98e6700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a4150ec0 con 0x7f98a415c470 2022-11-27T15:14:55.068 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a98e6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41f4ac0 con 0x7f98a40ce6d0 2022-11-27T15:14:55.068 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a98e6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40c7790 con 0x7f98a40ceb50 2022-11-27T15:14:55.068 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a37fe700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a415c470 0x7f98a41f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:55.069 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a37fe700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a415c470 0x7f98a41f08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39882/0 (socket says 172.21.15.35:39882) 2022-11-27T15:14:55.069 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a37fe700 1 -- 172.21.15.35:0/286984952 learned_addr learned my addr 172.21.15.35:0/286984952 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:55.069 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a27fc700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98a40ceb50 0x7f98a41f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:55.069 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a2ffd700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98a40ce6d0 0x7f98a4151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:55.070 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a37fe700 1 -- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98a40ceb50 msgr2=0x7f98a41f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:55.070 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a37fe700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98a40ceb50 0x7f98a41f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:55.070 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a37fe700 1 -- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98a40ce6d0 msgr2=0x7f98a4151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:55.070 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a37fe700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98a40ce6d0 0x7f98a4151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:55.071 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a37fe700 1 -- 172.21.15.35:0/286984952 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c009b30 con 0x7f98a415c470 2022-11-27T15:14:55.071 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a2ffd700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98a40ce6d0 0x7f98a4151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T15:14:55.071 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f98a37fe700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a415c470 0x7f98a41f08c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f98a4071850 tx=0x7f989c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:55.071 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.057+0000 7f988bfff700 1 -- 172.21.15.35:0/286984952 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c0099b0 con 0x7f98a415c470 2022-11-27T15:14:55.072 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.058+0000 7f988bfff700 1 -- 172.21.15.35:0/286984952 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c0092e0 con 0x7f98a415c470 2022-11-27T15:14:55.072 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.058+0000 7f988bfff700 1 -- 172.21.15.35:0/286984952 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c01c640 con 0x7f98a415c470 2022-11-27T15:14:55.072 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.058+0000 7f98a98e6700 1 -- 172.21.15.35:0/286984952 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a40c7a70 con 0x7f98a415c470 2022-11-27T15:14:55.072 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.058+0000 7f98a98e6700 1 -- 172.21.15.35:0/286984952 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a41f4750 con 0x7f98a415c470 2022-11-27T15:14:55.072 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.058+0000 7f98a98e6700 1 -- 172.21.15.35:0/286984952 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a404e190 con 0x7f98a415c470 2022-11-27T15:14:55.073 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.059+0000 7f988bfff700 1 -- 172.21.15.35:0/286984952 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f989c022030 con 0x7f98a415c470 2022-11-27T15:14:55.073 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.059+0000 7f988bfff700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9884066590 0x7f9884068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:55.073 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.059+0000 7f988bfff700 1 -- 172.21.15.35:0/286984952 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f989c084d10 con 0x7f98a415c470 2022-11-27T15:14:55.073 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.059+0000 7f98a2ffd700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9884066590 0x7f9884068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:55.074 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.060+0000 7f98a2ffd700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9884066590 0x7f9884068a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9898000f60 tx=0x7f9898009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:55.074 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.064+0000 7f988bfff700 1 -- 172.21.15.35:0/286984952 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f989c052770 con 0x7f98a415c470 2022-11-27T15:14:55.256 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:55.255+0000 7f98a98e6700 1 -- 172.21.15.35:0/286984952 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f98a40cca10 con 0x7f98a415c470 2022-11-27T15:14:55.298 DEBUG:teuthology.orchestra.run.smithi035:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T15:14:55.301 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T15:14:56.100 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.098+0000 7f988bfff700 1 -- 172.21.15.35:0/286984952 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f989c055d90 con 0x7f98a415c470 2022-11-27T15:14:56.104 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.102+0000 7f98a98e6700 1 -- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9884066590 msgr2=0x7f9884068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:56.104 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.102+0000 7f98a98e6700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9884066590 0x7f9884068a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9898000f60 tx=0x7f9898009040 comp rx=0 tx=0).stop 2022-11-27T15:14:56.104 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.102+0000 7f98a98e6700 1 -- 172.21.15.35:0/286984952 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a415c470 msgr2=0x7f98a41f08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:56.105 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.102+0000 7f98a98e6700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a415c470 0x7f98a41f08c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f98a4071850 tx=0x7f989c004ad0 comp rx=0 tx=0).stop 2022-11-27T15:14:56.105 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.103+0000 7f98a98e6700 1 -- 172.21.15.35:0/286984952 shutdown_connections 2022-11-27T15:14:56.106 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.103+0000 7f98a98e6700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7f9884066590 0x7f9884068a40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:56.106 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.103+0000 7f98a98e6700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f98a415c470 0x7f98a41f08c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:56.106 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.103+0000 7f98a98e6700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98a40ce6d0 0x7f98a4151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:56.106 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.103+0000 7f98a98e6700 1 --2- 172.21.15.35:0/286984952 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98a40ceb50 0x7f98a41f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:56.107 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.103+0000 7f98a98e6700 1 -- 172.21.15.35:0/286984952 >> 172.21.15.35:0/286984952 conn(0x7f98a4070b40 msgr2=0x7f98a415c840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:56.107 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.103+0000 7f98a98e6700 1 -- 172.21.15.35:0/286984952 shutdown_connections 2022-11-27T15:14:56.107 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.103+0000 7f98a98e6700 1 -- 172.21.15.35:0/286984952 wait complete. 2022-11-27T15:14:56.108 INFO:teuthology.orchestra.run.smithi035.stderr:pool 'cephfs_metadata' created 2022-11-27T15:14:56.122 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2022-11-27T15:14:56.262 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.260+0000 7efe7e631700 1 Processor -- start 2022-11-27T15:14:56.262 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.260+0000 7efe7e631700 1 -- start start 2022-11-27T15:14:56.265 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe7e631700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7efe78155e20 0x7efe780cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:56.265 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe7e631700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 0x7efe780b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:56.266 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe7e631700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe780b1060 con 0x7efe780b0c00 2022-11-27T15:14:56.266 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe7e631700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe780cd1a0 con 0x7efe78155e20 2022-11-27T15:14:56.266 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe7e631700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7815be10 con 0x7efe780ccd50 2022-11-27T15:14:56.266 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe77fff700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7efe78155e20 0x7efe780cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:56.267 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe77fff700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7efe78155e20 0x7efe780cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39886/0 (socket says 172.21.15.35:39886) 2022-11-27T15:14:56.267 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe77fff700 1 -- 172.21.15.35:0/3104360350 learned_addr learned my addr 172.21.15.35:0/3104360350 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:56.267 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe777fe700 1 --2- 172.21.15.35:0/3104360350 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 0x7efe780b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:56.267 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe77fff700 1 -- 172.21.15.35:0/3104360350 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 msgr2=0x7efe780b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:56.268 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe77fff700 1 --2- 172.21.15.35:0/3104360350 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 0x7efe780b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:56.268 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe77fff700 1 -- 172.21.15.35:0/3104360350 >> v1:172.21.15.85:6789/0 conn(0x7efe780b0c00 legacy=0x7efe7806e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:56.268 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe77fff700 1 -- 172.21.15.35:0/3104360350 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe780cd380 con 0x7efe78155e20 2022-11-27T15:14:56.268 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.261+0000 7efe77fff700 1 --2- 172.21.15.35:0/3104360350 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe78155e20 0x7efe780cc810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efe680098f0 tx=0x7efe68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8098b0554e96e2e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:56.268 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe76ffd700 1 -- 172.21.15.35:0/3104360350 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68004230 con 0x7efe78155e20 2022-11-27T15:14:56.269 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe76ffd700 1 -- 172.21.15.35:0/3104360350 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe68004390 con 0x7efe78155e20 2022-11-27T15:14:56.269 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe76ffd700 1 -- 172.21.15.35:0/3104360350 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe680036a0 con 0x7efe78155e20 2022-11-27T15:14:56.269 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe7e631700 1 -- 172.21.15.35:0/3104360350 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe78155e20 msgr2=0x7efe780cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:56.269 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe7e631700 1 --2- 172.21.15.35:0/3104360350 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe78155e20 0x7efe780cc810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efe680098f0 tx=0x7efe68009c00 comp rx=0 tx=0).stop 2022-11-27T15:14:56.270 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe7e631700 1 -- 172.21.15.35:0/3104360350 shutdown_connections 2022-11-27T15:14:56.270 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe7e631700 1 --2- 172.21.15.35:0/3104360350 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe78155e20 0x7efe780cc810 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:56.270 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe7e631700 1 --2- 172.21.15.35:0/3104360350 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 0x7efe780b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:56.270 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe7e631700 1 -- 172.21.15.35:0/3104360350 >> 172.21.15.35:0/3104360350 conn(0x7efe7806d9f0 msgr2=0x7efe780cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:56.271 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe7e631700 1 -- 172.21.15.35:0/3104360350 shutdown_connections 2022-11-27T15:14:56.271 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.262+0000 7efe7e631700 1 -- 172.21.15.35:0/3104360350 wait complete. 2022-11-27T15:14:56.271 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.263+0000 7efe7e631700 1 Processor -- start 2022-11-27T15:14:56.271 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.264+0000 7efe7e631700 1 -- start start 2022-11-27T15:14:56.275 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.264+0000 7efe7e631700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe780b0c00 0x7efe78150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:56.275 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.264+0000 7efe7e631700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 0x7efe78151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:56.275 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.264+0000 7efe7e631700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efe78155e20 0x7efe78069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:56.275 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.264+0000 7efe7e631700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe780abd60 con 0x7efe780b0c00 2022-11-27T15:14:56.276 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.264+0000 7efe7e631700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7814e910 con 0x7efe78155e20 2022-11-27T15:14:56.276 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.264+0000 7efe7e631700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7814ee90 con 0x7efe780ccd50 2022-11-27T15:14:56.276 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe77fff700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe780b0c00 0x7efe78150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:56.276 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe77fff700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe780b0c00 0x7efe78150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39892/0 (socket says 172.21.15.35:39892) 2022-11-27T15:14:56.277 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe77fff700 1 -- 172.21.15.35:0/1591146073 learned_addr learned my addr 172.21.15.35:0/1591146073 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:56.277 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe777fe700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 0x7efe78151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:56.277 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe7cbce700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efe78155e20 0x7efe78069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:56.277 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe77fff700 1 -- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 msgr2=0x7efe78151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:56.278 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe77fff700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 0x7efe78151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:56.278 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe77fff700 1 -- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efe78155e20 msgr2=0x7efe78069f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:56.278 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe77fff700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efe78155e20 0x7efe78069f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:56.278 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe77fff700 1 -- 172.21.15.35:0/1591146073 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe68009550 con 0x7efe780b0c00 2022-11-27T15:14:56.278 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe777fe700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 0x7efe78151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T15:14:56.279 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe77fff700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe780b0c00 0x7efe78150d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe78070590 tx=0x7efe68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:56.279 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe74ff9700 1 -- 172.21.15.35:0/1591146073 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6800b860 con 0x7efe780b0c00 2022-11-27T15:14:56.279 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe74ff9700 1 -- 172.21.15.35:0/1591146073 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6800b9c0 con 0x7efe780b0c00 2022-11-27T15:14:56.279 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe7e631700 1 -- 172.21.15.35:0/1591146073 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe78151260 con 0x7efe780b0c00 2022-11-27T15:14:56.280 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe74ff9700 1 -- 172.21.15.35:0/1591146073 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6800bd70 con 0x7efe780b0c00 2022-11-27T15:14:56.280 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.265+0000 7efe7e631700 1 -- 172.21.15.35:0/1591146073 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe780aba80 con 0x7efe780b0c00 2022-11-27T15:14:56.280 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.266+0000 7efe7e631700 1 -- 172.21.15.35:0/1591146073 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7804e190 con 0x7efe780b0c00 2022-11-27T15:14:56.280 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.267+0000 7efe74ff9700 1 -- 172.21.15.35:0/1591146073 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7efe68017020 con 0x7efe780b0c00 2022-11-27T15:14:56.280 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.267+0000 7efe74ff9700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7efe5c066590 0x7efe5c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:56.281 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.267+0000 7efe74ff9700 1 -- 172.21.15.35:0/1591146073 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5576+0+0 (secure 0 0 0) 0x7efe68084e80 con 0x7efe780b0c00 2022-11-27T15:14:56.281 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.267+0000 7efe777fe700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7efe5c066590 0x7efe5c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:56.281 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.268+0000 7efe777fe700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7efe5c066590 0x7efe5c068a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efe60006fd0 tx=0x7efe60008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:56.281 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.272+0000 7efe74ff9700 1 -- 172.21.15.35:0/1591146073 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7efe68052770 con 0x7efe780b0c00 2022-11-27T15:14:56.466 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:56.464+0000 7efe7e631700 1 -- 172.21.15.35:0/1591146073 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7efe7814e6d0 con 0x7efe780b0c00 2022-11-27T15:14:57.112 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.110+0000 7efe74ff9700 1 -- 172.21.15.35:0/1591146073 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) v1 ==== 171+0+0 (secure 0 0 0) 0x7efe68055d90 con 0x7efe780b0c00 2022-11-27T15:14:57.117 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.116+0000 7efe7e631700 1 -- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7efe5c066590 msgr2=0x7efe5c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:57.118 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.116+0000 7efe7e631700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7efe5c066590 0x7efe5c068a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efe60006fd0 tx=0x7efe60008040 comp rx=0 tx=0).stop 2022-11-27T15:14:57.119 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.116+0000 7efe7e631700 1 -- 172.21.15.35:0/1591146073 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe780b0c00 msgr2=0x7efe78150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:57.119 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.116+0000 7efe7e631700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe780b0c00 0x7efe78150d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe78070590 tx=0x7efe68004750 comp rx=0 tx=0).stop 2022-11-27T15:14:57.120 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.116+0000 7efe7e631700 1 -- 172.21.15.35:0/1591146073 shutdown_connections 2022-11-27T15:14:57.120 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.116+0000 7efe7e631700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:6832/100969,v1:172.21.15.85:6833/100969] conn(0x7efe5c066590 0x7efe5c068a40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:57.121 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.116+0000 7efe7e631700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efe78155e20 0x7efe78069f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:57.121 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.116+0000 7efe7e631700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7efe780b0c00 0x7efe78150d20 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe78070590 tx=0x7efe68004750 comp rx=0 tx=0).stop 2022-11-27T15:14:57.122 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.116+0000 7efe7e631700 1 --2- 172.21.15.35:0/1591146073 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe780ccd50 0x7efe78151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:57.122 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.116+0000 7efe7e631700 1 -- 172.21.15.35:0/1591146073 >> 172.21.15.35:0/1591146073 conn(0x7efe7806d9f0 msgr2=0x7efe7815e740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:57.122 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.118+0000 7efe7e631700 1 -- 172.21.15.35:0/1591146073 shutdown_connections 2022-11-27T15:14:57.123 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.118+0000 7efe7e631700 1 -- 172.21.15.35:0/1591146073 wait complete. 2022-11-27T15:14:57.128 INFO:teuthology.orchestra.run.smithi035.stderr:pool 'cephfs_data' created 2022-11-27T15:14:57.145 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-11-27T15:14:57.279 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.277+0000 7f8927ab6700 1 Processor -- start 2022-11-27T15:14:57.280 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.277+0000 7f8927ab6700 1 -- start start 2022-11-27T15:14:57.285 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8927ab6700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f892015b5d0 0x7f89200c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:57.285 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8927ab6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f892006aaf0 0x7f89200cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:57.285 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8927ab6700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f892015f3c0 0x7f89200cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:57.285 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8927ab6700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8920070780 con 0x7f892015b5d0 2022-11-27T15:14:57.286 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8927ab6700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8920070a60 con 0x7f892015f3c0 2022-11-27T15:14:57.286 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8927ab6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8920070d40 con 0x7f892006aaf0 2022-11-27T15:14:57.286 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8925852700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f892015b5d0 0x7f89200c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:57.286 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8925852700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f892015b5d0 0x7f89200c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39900/0 (socket says 172.21.15.35:39900) 2022-11-27T15:14:57.287 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8925852700 1 -- 172.21.15.35:0/2117757514 learned_addr learned my addr 172.21.15.35:0/2117757514 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:57.287 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8926053700 1 --2- 172.21.15.35:0/2117757514 >> v2:172.21.15.85:3300/0 conn(0x7f892015f3c0 0x7f89200cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:57.287 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.278+0000 7f8925051700 1 --2- 172.21.15.35:0/2117757514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f892006aaf0 0x7f89200cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:57.287 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8925852700 1 -- 172.21.15.35:0/2117757514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f892006aaf0 msgr2=0x7f89200cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:57.287 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8925852700 1 --2- 172.21.15.35:0/2117757514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f892006aaf0 0x7f89200cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:57.288 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8925852700 1 -- 172.21.15.35:0/2117757514 >> v2:172.21.15.85:3300/0 conn(0x7f892015f3c0 msgr2=0x7f89200cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:14:57.288 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8925852700 1 --2- 172.21.15.35:0/2117757514 >> v2:172.21.15.85:3300/0 conn(0x7f892015f3c0 0x7f89200cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:57.288 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8925852700 1 -- 172.21.15.35:0/2117757514 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8920071020 con 0x7f892015b5d0 2022-11-27T15:14:57.288 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8925051700 1 --2- 172.21.15.35:0/2117757514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f892006aaf0 0x7f89200cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T15:14:57.289 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8926053700 1 --2- 172.21.15.35:0/2117757514 >> v2:172.21.15.85:3300/0 conn(0x7f892015f3c0 0x7f89200cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T15:14:57.289 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8925852700 1 --2- 172.21.15.35:0/2117757514 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f892015b5d0 0x7f89200c89e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89200ce3f0 tx=0x7f8910009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a419438e5cec288 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:14:57.289 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8924850700 1 -- 172.21.15.35:0/2117757514 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891000b870 con 0x7f892015b5d0 2022-11-27T15:14:57.289 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8924850700 1 -- 172.21.15.35:0/2117757514 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891000b9d0 con 0x7f892015b5d0 2022-11-27T15:14:57.290 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8924850700 1 -- 172.21.15.35:0/2117757514 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891000bd60 con 0x7f892015b5d0 2022-11-27T15:14:57.290 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8927ab6700 1 -- 172.21.15.35:0/2117757514 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f892015b5d0 msgr2=0x7f89200c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:14:57.290 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8927ab6700 1 --2- 172.21.15.35:0/2117757514 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f892015b5d0 0x7f89200c89e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89200ce3f0 tx=0x7f8910009bd0 comp rx=0 tx=0).stop 2022-11-27T15:14:57.290 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8927ab6700 1 -- 172.21.15.35:0/2117757514 shutdown_connections 2022-11-27T15:14:57.291 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8927ab6700 1 --2- 172.21.15.35:0/2117757514 >> v2:172.21.15.85:3300/0 conn(0x7f892015f3c0 0x7f89200cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:57.291 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.279+0000 7f8927ab6700 1 --2- 172.21.15.35:0/2117757514 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f892015b5d0 0x7f89200c89e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:57.291 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.280+0000 7f8927ab6700 1 --2- 172.21.15.35:0/2117757514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f892006aaf0 0x7f89200cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T15:14:57.291 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.280+0000 7f8927ab6700 1 -- 172.21.15.35:0/2117757514 >> 172.21.15.35:0/2117757514 conn(0x7f89200aff10 msgr2=0x7f89200b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:14:57.291 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.280+0000 7f8927ab6700 1 -- 172.21.15.35:0/2117757514 shutdown_connections 2022-11-27T15:14:57.292 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.280+0000 7f8927ab6700 1 -- 172.21.15.35:0/2117757514 wait complete. 2022-11-27T15:14:57.292 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.281+0000 7f8927ab6700 1 Processor -- start 2022-11-27T15:14:57.292 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.281+0000 7f8927ab6700 1 -- start start 2022-11-27T15:14:57.292 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8927ab6700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f892006aaf0 0x7f89200c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:57.293 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8927ab6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f892015b5d0 0x7f8920153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:57.293 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8927ab6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f892015f3c0 0x7f89200acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T15:14:57.293 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8927ab6700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89200b3050 con 0x7f892006aaf0 2022-11-27T15:14:57.293 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8927ab6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89200c9fc0 con 0x7f892015b5d0 2022-11-27T15:14:57.293 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8927ab6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8920153ec0 con 0x7f892015f3c0 2022-11-27T15:14:57.294 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8925852700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f892006aaf0 0x7f89200c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:57.294 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8925852700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f892006aaf0 0x7f89200c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39906/0 (socket says 172.21.15.35:39906) 2022-11-27T15:14:57.294 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8925852700 1 -- 172.21.15.35:0/3531196643 learned_addr learned my addr 172.21.15.35:0/3531196643 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-27T15:14:57.294 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8925051700 1 --2- 172.21.15.35:0/3531196643 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f892015b5d0 0x7f8920153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T15:14:57.295 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-27T15:14:57.282+0000 7f8926053700 1 --2- 172.21.15.35:0/3531196643 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f892015f3c0 0x7f89200acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=