2022-11-27T16:05:49.296 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-27T16:05:49.300 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} --archive /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088644 --verbose --owner scheduled_yuriw@teuthology --name yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi -- /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088644/orig.config.yaml 2022-11-27T16:05:49.379 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-11-27T16:05:49.479 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088644 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} email: ceph-qa@ceph.io first_in_suite: false job_id: '7088644' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - inconsistent rstat on inode - Error recovering journal sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 sleep_before_teardown: 0 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi134.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZKu7FffAcWUek5T7f+sB4QCRONCfIL/vSKhrwExRvmnF9N+QHTo8HIQr+bYBSmWQIkRiKZZEOuLgnd+S4W9kehQfVaeYjyA2p9noW72reaPThszR2sxKJLh9IFMcIlbDN1Xleo5mL25TdKbB6xDc42DO+DnOtvpotkSwE2etew+xzTt9kMQRNofF2jVkuHzYpCGA4liOZzQYsqK0pc/QUnqcxLP/YS6KDl57n53bl1ltO9shTTIZ4xhq3tsQtqT5JBd3sLyll5+PGl2u2UW1IgHKy+dU2OfXEzrBMosy6Bi1Ud72up8wkrYiLaRgst3GYE7tqe8cLcs8xDkF2749gvjPwREY6ptGjESOM0LHLTXn10DqVxjYmOY/zFqX0t+xCjvHszFh/yyzssdTJk/9rcCMMw0AR3BM0pIKYRAV96i8wMK/I7rvWSuk0kDzW1uqSQWgGjklOIbWwepMvBZb0sDOzoqgYO7w/4VAfOZm6z9NIxACxpKSDDb80Rn8ZAAM= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLA9R28odf0ze2bQSa9JOMXiJ+7o1/SEEkiimPC5DzhKKgL0I9ZUMOMUuuI64CwctuxdvNZpN/4fGxDb+sMQAiA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_data_scan teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-23_17:24:26 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23126 2022-11-27T16:05:49.479 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa; will attempt to use it 2022-11-27T16:05:49.480 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa/tasks 2022-11-27T16:05:49.481 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-11-27T16:05:49.483 INFO:teuthology.task.internal:Checking packages... 2022-11-27T16:05:49.509 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '697b0d99d9a52df92b20c8e053dcd46c75f364a9' 2022-11-27T16:05:49.509 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T16:05:49.510 INFO:teuthology.packaging:ref: None 2022-11-27T16:05:49.510 INFO:teuthology.packaging:tag: None 2022-11-27T16:05:49.510 INFO:teuthology.packaging:branch: main 2022-11-27T16:05:49.511 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:05:49.511 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=main 2022-11-27T16:05:49.762 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-982.g2bb1aa26 2022-11-27T16:05:49.763 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-11-27T16:05:49.773 INFO:teuthology.task.internal:no buildpackages task found 2022-11-27T16:05:49.773 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-11-27T16:05:49.803 INFO:teuthology.task.internal:Saving configuration 2022-11-27T16:05:49.821 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-11-27T16:05:49.870 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-27T16:05:49.896 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088644', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 16:00:49.247987', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM1YrKo9OoJ59qd1DPA828HGjm3SL0JGvdMXtnAeAeV2mfkVibJ9ORxmlcAyO4QphE/8NKrzqNSqWS9yBQ42Oqw='} 2022-11-27T16:05:49.926 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088644', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 16:00:49.246383', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLA9R28odf0ze2bQSa9JOMXiJ+7o1/SEEkiimPC5DzhKKgL0I9ZUMOMUuuI64CwctuxdvNZpN/4fGxDb+sMQAiA='} 2022-11-27T16:05:49.926 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-11-27T16:05:49.944 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-11-27T16:05:49.945 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-11-27T16:05:49.945 INFO:teuthology.run_tasks:Running task console_log... 2022-11-27T16:05:50.049 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f54ab33c840>, signals=[15]) 2022-11-27T16:05:50.050 INFO:teuthology.run_tasks:Running task internal.connect... 2022-11-27T16:05:50.074 INFO:teuthology.task.internal:Opening connections... 2022-11-27T16:05:50.075 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2022-11-27T16:05:50.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:05:50.162 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2022-11-27T16:05:50.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:05:50.261 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-11-27T16:05:50.268 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2022-11-27T16:05:50.290 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2022-11-27T16:05:50.290 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2022-11-27T16:05:50.349 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="Red Hat Enterprise Linux" 2022-11-27T16:05:50.350 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="8.6 (Ootpa)" 2022-11-27T16:05:50.350 INFO:teuthology.orchestra.run.smithi134.stdout:ID="rhel" 2022-11-27T16:05:50.350 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="fedora" 2022-11-27T16:05:50.351 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="8.6" 2022-11-27T16:05:50.351 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el8" 2022-11-27T16:05:50.351 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-11-27T16:05:50.351 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2022-11-27T16:05:50.352 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-11-27T16:05:50.352 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://www.redhat.com/" 2022-11-27T16:05:50.352 INFO:teuthology.orchestra.run.smithi134.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-11-27T16:05:50.352 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T16:05:50.353 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:05:50.353 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T16:05:50.353 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-11-27T16:05:50.353 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-11-27T16:05:50.353 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-11-27T16:05:50.354 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2022-11-27T16:05:50.375 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2022-11-27T16:05:50.396 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2022-11-27T16:05:50.397 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2022-11-27T16:05:50.456 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Red Hat Enterprise Linux" 2022-11-27T16:05:50.456 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8.6 (Ootpa)" 2022-11-27T16:05:50.457 INFO:teuthology.orchestra.run.smithi191.stdout:ID="rhel" 2022-11-27T16:05:50.457 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="fedora" 2022-11-27T16:05:50.457 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8.6" 2022-11-27T16:05:50.457 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2022-11-27T16:05:50.458 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-11-27T16:05:50.458 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2022-11-27T16:05:50.458 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-11-27T16:05:50.458 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.redhat.com/" 2022-11-27T16:05:50.459 INFO:teuthology.orchestra.run.smithi191.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-11-27T16:05:50.459 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T16:05:50.459 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:05:50.459 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T16:05:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-11-27T16:05:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-11-27T16:05:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-11-27T16:05:50.460 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2022-11-27T16:05:50.483 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-11-27T16:05:50.494 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-11-27T16:05:50.519 INFO:teuthology.task.internal:Checking for old test directory... 2022-11-27T16:05:50.520 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2022-11-27T16:05:50.523 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2022-11-27T16:05:50.542 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-11-27T16:05:50.551 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-11-27T16:05:50.552 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2022-11-27T16:05:50.582 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2022-11-27T16:05:50.648 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-11-27T16:05:50.713 INFO:teuthology.run_tasks:Running task kernel... 2022-11-27T16:05:50.735 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2022-11-27T16:05:50.736 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2022-11-27T16:05:50.737 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2022-11-27T16:05:50.737 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T16:05:50.738 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T16:05:50.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:05:50.760 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2022-11-27T16:05:50.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:05:50.764 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2022-11-27T16:05:50.819 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-372.9.1.el8.x86_64 2022-11-27T16:05:50.819 INFO:teuthology.task.kernel:Running kernel on smithi191: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:05:50.820 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2022-11-27T16:05:50.822 INFO:teuthology.orchestra.run.smithi134.stdout:4.18.0-372.9.1.el8.x86_64 2022-11-27T16:05:50.822 INFO:teuthology.task.kernel:Running kernel on smithi134: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:05:50.823 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum install -y kernel 2022-11-27T16:05:51.790 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2022-11-27T16:05:51.790 INFO:teuthology.orchestra.run.smithi134.stdout:Unable to read consumer identity 2022-11-27T16:05:51.791 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2022-11-27T16:05:51.792 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2022-11-27T16:05:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:05:51.903 INFO:teuthology.orchestra.run.smithi134.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:05:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:05:51.904 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:05:51.905 INFO:teuthology.orchestra.run.smithi191.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:05:51.905 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:05:54.980 INFO:teuthology.orchestra.run.smithi134.stdout:Copr repo for python3-asyncssh owned by ceph 1.7 kB/s | 3.5 kB 00:01 2022-11-27T16:05:55.236 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 1.8 kB/s | 3.5 kB 00:01 2022-11-27T16:05:55.957 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2022-11-27T16:05:56.187 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2022-11-27T16:05:58.939 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 371 kB/s | 24 kB 00:00 2022-11-27T16:05:59.221 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 215 kB/s | 24 kB 00:00 2022-11-27T16:05:59.592 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-11-27T16:05:59.645 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2022-11-27T16:05:59.646 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2022-11-27T16:05:59.647 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2022-11-27T16:05:59.783 DEBUG:teuthology.orchestra.run.smithi134:> echo no | sudo yum reinstall kernel || true 2022-11-27T16:05:59.899 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-11-27T16:05:59.956 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2022-11-27T16:05:59.957 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2022-11-27T16:05:59.958 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2022-11-27T16:06:00.107 DEBUG:teuthology.orchestra.run.smithi191:> echo no | sudo yum reinstall kernel || true 2022-11-27T16:06:00.117 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2022-11-27T16:06:00.118 INFO:teuthology.orchestra.run.smithi134.stdout:Unable to read consumer identity 2022-11-27T16:06:00.164 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:06:00.165 INFO:teuthology.orchestra.run.smithi134.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:06:00.165 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:06:00.405 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 04:05:58 PM UTC. 2022-11-27T16:06:00.440 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2022-11-27T16:06:00.441 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2022-11-27T16:06:00.451 INFO:teuthology.orchestra.run.smithi134.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:06:00.515 INFO:teuthology.orchestra.run.smithi134.stderr:Error: No packages marked for reinstall. 2022-11-27T16:06:00.548 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:06:00.549 INFO:teuthology.orchestra.run.smithi191.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:06:00.549 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:06:00.563 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum reinstall -y kernel || true 2022-11-27T16:06:00.795 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:01 ago on Sun 27 Nov 2022 04:05:59 PM UTC. 2022-11-27T16:06:00.835 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:06:00.884 INFO:teuthology.orchestra.run.smithi191.stderr:Error: No packages marked for reinstall. 2022-11-27T16:06:00.894 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2022-11-27T16:06:00.894 INFO:teuthology.orchestra.run.smithi134.stdout:Unable to read consumer identity 2022-11-27T16:06:00.931 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum reinstall -y kernel || true 2022-11-27T16:06:00.936 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:06:00.937 INFO:teuthology.orchestra.run.smithi134.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:06:00.937 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:06:01.172 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:03 ago on Sun 27 Nov 2022 04:05:58 PM UTC. 2022-11-27T16:06:01.212 INFO:teuthology.orchestra.run.smithi134.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:06:01.255 INFO:teuthology.orchestra.run.smithi134.stderr:Error: No packages marked for reinstall. 2022-11-27T16:06:01.260 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2022-11-27T16:06:01.261 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2022-11-27T16:06:01.301 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel --last | head -n 1 2022-11-27T16:06:01.305 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:06:01.305 INFO:teuthology.orchestra.run.smithi191.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:06:01.305 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:06:01.381 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-11-27T16:06:01.382 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:06:01.382 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T16:06:01.383 DEBUG:teuthology.parallel:result is None 2022-11-27T16:06:01.542 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 04:05:59 PM UTC. 2022-11-27T16:06:01.581 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:06:01.620 INFO:teuthology.orchestra.run.smithi191.stderr:Error: No packages marked for reinstall. 2022-11-27T16:06:01.666 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel --last | head -n 1 2022-11-27T16:06:01.745 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-11-27T16:06:01.747 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:06:01.747 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T16:06:01.747 DEBUG:teuthology.parallel:result is None 2022-11-27T16:06:01.748 INFO:teuthology.run_tasks:Running task internal.base... 2022-11-27T16:06:01.761 INFO:teuthology.task.internal:Creating test directory... 2022-11-27T16:06:01.761 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T16:06:01.765 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T16:06:01.781 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-11-27T16:06:01.802 INFO:teuthology.run_tasks:Running task internal.archive... 2022-11-27T16:06:01.837 INFO:teuthology.task.internal:Creating archive directory... 2022-11-27T16:06:01.838 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T16:06:01.841 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T16:06:01.872 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-11-27T16:06:01.881 INFO:teuthology.task.internal:Enabling coredump saving... 2022-11-27T16:06:01.881 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T16:06:01.905 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T16:06:01.938 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:06:01.940 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:06:01.952 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:06:01.955 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:06:01.957 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-11-27T16:06:01.964 INFO:teuthology.task.internal:Configuring sudo... 2022-11-27T16:06:01.965 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T16:06:01.996 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T16:06:02.023 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-11-27T16:06:02.032 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-11-27T16:06:02.033 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T16:06:02.066 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T16:06:02.083 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:06:02.165 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:06:02.219 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:06:02.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:06:02.345 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:06:02.346 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T16:06:02.413 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:06:02.464 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:06:02.512 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:06:02.597 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:06:02.638 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:06:02.639 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T16:06:02.705 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2022-11-27T16:06:02.709 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2022-11-27T16:06:02.754 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T16:06:02.802 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T16:06:03.132 INFO:teuthology.run_tasks:Running task internal.timer... 2022-11-27T16:06:03.141 INFO:teuthology.task.internal:Starting timer... 2022-11-27T16:06:03.141 INFO:teuthology.run_tasks:Running task pcp... 2022-11-27T16:06:03.180 INFO:teuthology.run_tasks:Running task selinux... 2022-11-27T16:06:03.222 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2022-11-27T16:06:03.223 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2022-11-27T16:06:03.278 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2022-11-27T16:06:03.280 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2022-11-27T16:06:03.338 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2022-11-27T16:06:03.340 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-11-27T16:06:03.340 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2022-11-27T16:06:03.364 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2022-11-27T16:06:03.364 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2022-11-27T16:06:03.384 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2022-11-27T16:06:03.385 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi134.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2022-11-27T16:06:03.386 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T16:06:03.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:06:03.433 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T16:06:03.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:06:03.462 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-11-27T16:06:03.463 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2022-11-27T16:06:03.501 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2022-11-27T16:06:03.530 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-11-27T16:06:03.541 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2022-11-27T16:06:03.541 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-11-27T16:06:03.567 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-11-27T16:06:03.569 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi134.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-11-27T16:14:05.311 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi134.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2022-11-27T16:14:05.339 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2022-11-27T16:14:05.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:14:05.432 DEBUG:teuthology.orchestra.run.smithi134:> true 2022-11-27T16:14:05.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2022-11-27T16:14:05.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:14:05.537 DEBUG:teuthology.orchestra.run.smithi191:> true 2022-11-27T16:14:05.605 INFO:teuthology.run_tasks:Running task clock... 2022-11-27T16:14:05.616 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-11-27T16:14:05.617 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:14:05.619 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:14:05.622 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:14:05.622 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:14:05.655 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:14:05.679 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:14:05.696 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:14:05.749 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2022-11-27T16:14:05.766 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2022-11-27T16:14:05.767 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2022-11-27T16:14:05.773 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2022-11-27T16:14:05.775 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:14:05.788 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:14:05.790 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:14:05.802 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:14:05.861 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2022-11-27T16:14:05.866 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:14:05.866 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2022-11-27T16:14:05.867 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:14:05.867 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:14:05.867 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:14:05.868 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:14:05.879 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2022-11-27T16:14:05.883 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:14:05.883 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2022-11-27T16:14:05.883 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:14:05.884 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:14:05.884 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:14:05.884 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:14:05.885 INFO:teuthology.run_tasks:Running task install... 2022-11-27T16:14:05.899 DEBUG:teuthology.task.install:project ceph 2022-11-27T16:14:05.901 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}} 2022-11-27T16:14:05.902 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'} 2022-11-27T16:14:05.902 INFO:teuthology.task.install:Using flavor: default 2022-11-27T16:14:05.918 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-11-27T16:14:05.919 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2022-11-27T16:14:05.920 DEBUG:teuthology.orchestra.run.smithi134:> sudo lsb_release -is 2022-11-27T16:14:05.961 INFO:teuthology.orchestra.run.smithi134.stdout:RedHatEnterprise 2022-11-27T16:14:05.963 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:14:05.963 DEBUG:teuthology.orchestra.run.smithi191:> sudo lsb_release -is 2022-11-27T16:14:05.964 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:14:05.965 DEBUG:teuthology.orchestra.run.smithi134:> sudo lsb_release -is 2022-11-27T16:14:06.006 INFO:teuthology.orchestra.run.smithi191.stdout:RedHatEnterprise 2022-11-27T16:14:06.008 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:14:06.009 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:14:06.010 DEBUG:teuthology.orchestra.run.smithi191:> sudo lsb_release -is 2022-11-27T16:14:06.042 INFO:teuthology.orchestra.run.smithi134.stdout:RedHatEnterprise 2022-11-27T16:14:06.044 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:14:06.044 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:14:06.110 INFO:teuthology.orchestra.run.smithi191.stdout:RedHatEnterprise 2022-11-27T16:14:06.111 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:14:06.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:14:06.219 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T16:14:06.219 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T16:14:06.257 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T16:14:06.258 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T16:14:06.341 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:14:06.341 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:14:06.342 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:14:06.375 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T16:14:06.375 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:14:06.377 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:14:06.378 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:14:06.378 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:14:06.406 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T16:14:06.407 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:14:06.443 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:14:06.475 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:14:06.520 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:14:06.553 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:14:06.586 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2022-11-27T16:14:06.588 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2022-11-27T16:14:06.619 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2022-11-27T16:14:06.621 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2022-11-27T16:14:06.959 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2022-11-27T16:14:06.991 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2022-11-27T16:14:07.325 INFO:teuthology.orchestra.run.smithi134.stdout:54 files removed 2022-11-27T16:14:07.345 INFO:teuthology.orchestra.run.smithi191.stdout:54 files removed 2022-11-27T16:14:07.373 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T16:14:07.392 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T16:14:07.703 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2022-11-27T16:14:07.721 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2022-11-27T16:14:08.291 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 295 kB/s | 81 kB 00:00 2022-11-27T16:14:08.310 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 297 kB/s | 81 kB 00:00 2022-11-27T16:14:08.583 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 36 kB/s | 9.4 kB 00:00 2022-11-27T16:14:08.603 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 36 kB/s | 9.4 kB 00:00 2022-11-27T16:14:08.851 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 6.1 kB/s | 1.5 kB 00:00 2022-11-27T16:14:08.862 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 6.4 kB/s | 1.5 kB 00:00 2022-11-27T16:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2022-11-27T16:14:11.479 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2022-11-27T16:14:13.912 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 5.4 MB/s | 13 MB 00:02 2022-11-27T16:14:15.826 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 3.0 MB/s | 13 MB 00:04 2022-11-27T16:14:16.879 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 447 kB/s | 24 kB 00:00 2022-11-27T16:14:17.715 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 69 MB/s | 53 MB 00:00 2022-11-27T16:14:18.768 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 412 kB/s | 24 kB 00:00 2022-11-27T16:14:19.539 INFO:teuthology.orchestra.run.smithi134.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 68 MB/s | 48 MB 00:00 2022-11-27T16:14:25.732 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 48 MB 00:00 2022-11-27T16:14:27.489 INFO:teuthology.orchestra.run.smithi134.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 71 MB/s | 53 MB 00:00 2022-11-27T16:14:36.783 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:14:36.784 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:14:36.799 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-11-27T16:14:36.800 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-11-27T16:14:36.801 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-11-27T16:14:36.802 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T16:14:36.804 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T16:14:36.808 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T16:14:36.809 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T16:14:36.940 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2022-11-27T16:14:36.953 INFO:teuthology.orchestra.run.smithi191.stdout:=========================================================================================================================== 2022-11-27T16:14:36.953 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2022-11-27T16:14:36.953 INFO:teuthology.orchestra.run.smithi191.stdout:=========================================================================================================================== 2022-11-27T16:14:36.954 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2022-11-27T16:14:36.954 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-11-27T16:14:36.954 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-11-27T16:14:36.954 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T16:14:36.955 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T16:14:36.955 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T16:14:36.955 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T16:14:36.956 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T16:14:36.956 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T16:14:36.956 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T16:14:36.956 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T16:14:36.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T16:14:36.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T16:14:36.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T16:14:36.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T16:14:36.958 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T16:14:36.958 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T16:14:36.958 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T16:14:36.958 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-11-27T16:14:36.959 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-11-27T16:14:36.959 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-11-27T16:14:36.959 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T16:14:36.959 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T16:14:36.960 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T16:14:36.960 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-11-27T16:14:36.960 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-11-27T16:14:36.960 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-11-27T16:14:36.961 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-11-27T16:14:36.961 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-11-27T16:14:36.961 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-11-27T16:14:36.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T16:14:36.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T16:14:36.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T16:14:36.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T16:14:36.962 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T16:14:36.963 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T16:14:36.963 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T16:14:36.963 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-11-27T16:14:36.963 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2022-11-27T16:14:36.964 INFO:teuthology.orchestra.run.smithi191.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-11-27T16:14:36.964 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:14:36.964 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:14:36.964 INFO:teuthology.orchestra.run.smithi191.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-11-27T16:14:36.965 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-11-27T16:14:36.965 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2022-11-27T16:14:36.965 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-11-27T16:14:36.965 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T16:14:36.966 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T16:14:36.966 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T16:14:36.966 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T16:14:36.966 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T16:14:36.967 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T16:14:36.967 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T16:14:36.967 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T16:14:36.968 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-11-27T16:14:36.968 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:14:36.968 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-11-27T16:14:36.968 INFO:teuthology.orchestra.run.smithi191.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-11-27T16:14:36.969 INFO:teuthology.orchestra.run.smithi191.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-11-27T16:14:36.969 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T16:14:36.969 INFO:teuthology.orchestra.run.smithi191.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-11-27T16:14:36.969 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-11-27T16:14:36.970 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:14:36.970 INFO:teuthology.orchestra.run.smithi191.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-11-27T16:14:36.970 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-11-27T16:14:36.970 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-11-27T16:14:36.971 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T16:14:36.971 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-11-27T16:14:36.971 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T16:14:36.971 INFO:teuthology.orchestra.run.smithi191.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-11-27T16:14:36.972 INFO:teuthology.orchestra.run.smithi191.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-11-27T16:14:36.972 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:14:36.972 INFO:teuthology.orchestra.run.smithi191.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-11-27T16:14:36.972 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-11-27T16:14:36.973 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-11-27T16:14:36.973 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-11-27T16:14:36.973 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-11-27T16:14:36.973 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-11-27T16:14:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-11-27T16:14:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-11-27T16:14:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T16:14:36.974 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-11-27T16:14:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T16:14:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T16:14:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T16:14:36.975 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T16:14:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T16:14:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T16:14:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-11-27T16:14:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T16:14:36.976 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T16:14:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-11-27T16:14:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T16:14:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T16:14:36.977 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T16:14:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-11-27T16:14:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-11-27T16:14:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-11-27T16:14:36.978 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:14:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T16:14:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T16:14:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-11-27T16:14:36.979 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-11-27T16:14:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T16:14:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T16:14:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-11-27T16:14:36.980 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-11-27T16:14:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-11-27T16:14:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T16:14:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T16:14:36.982 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-11-27T16:14:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-11-27T16:14:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-11-27T16:14:36.983 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-11-27T16:14:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-11-27T16:14:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-11-27T16:14:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T16:14:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:14:36.984 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T16:14:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T16:14:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-11-27T16:14:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T16:14:36.985 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T16:14:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T16:14:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T16:14:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T16:14:36.986 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T16:14:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T16:14:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T16:14:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-11-27T16:14:36.987 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T16:14:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-11-27T16:14:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T16:14:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-11-27T16:14:36.988 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-11-27T16:14:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-11-27T16:14:36.989 INFO:teuthology.orchestra.run.smithi191.stdout:Enabling module streams: 2022-11-27T16:14:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql 10 2022-11-27T16:14:36.989 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:14:36.990 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2022-11-27T16:14:36.990 INFO:teuthology.orchestra.run.smithi191.stdout:=========================================================================================================================== 2022-11-27T16:14:36.990 INFO:teuthology.orchestra.run.smithi191.stdout:Install 128 Packages 2022-11-27T16:14:36.990 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 5 Packages 2022-11-27T16:14:36.991 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:14:36.991 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 219 M 2022-11-27T16:14:36.992 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2022-11-27T16:14:37.098 INFO:teuthology.orchestra.run.smithi191.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 49 kB/s | 6.4 kB 00:00 2022-11-27T16:14:37.349 INFO:teuthology.orchestra.run.smithi191.stdout:(2/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 3.3 MB/s | 845 kB 00:00 2022-11-27T16:14:37.424 INFO:teuthology.orchestra.run.smithi191.stdout:(3/133): ceph-immutable-object-cache-18.0.0-594 2.2 MB/s | 165 kB 00:00 2022-11-27T16:14:37.783 INFO:teuthology.orchestra.run.smithi191.stdout:(4/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 6.3 MB/s | 2.3 MB 00:00 2022-11-27T16:14:37.850 INFO:teuthology.orchestra.run.smithi191.stdout:(5/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 5.9 MB/s | 5.2 MB 00:00 2022-11-27T16:14:38.117 INFO:teuthology.orchestra.run.smithi191.stdout:(6/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 4.6 MB/s | 1.5 MB 00:00 2022-11-27T16:14:38.292 INFO:teuthology.orchestra.run.smithi191.stdout:(7/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 8.7 MB/s | 3.9 MB 00:00 2022-11-27T16:14:38.522 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:14:38.522 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:14:38.537 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-11-27T16:14:38.539 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-11-27T16:14:38.540 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-11-27T16:14:38.541 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T16:14:38.543 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T16:14:38.547 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T16:14:38.548 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T16:14:38.682 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2022-11-27T16:14:38.695 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2022-11-27T16:14:38.695 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2022-11-27T16:14:38.696 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2022-11-27T16:14:38.696 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2022-11-27T16:14:38.696 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-11-27T16:14:38.696 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-11-27T16:14:38.697 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T16:14:38.697 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T16:14:38.697 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T16:14:38.698 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T16:14:38.698 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T16:14:38.698 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T16:14:38.698 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T16:14:38.698 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T16:14:38.699 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T16:14:38.699 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T16:14:38.699 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T16:14:38.699 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T16:14:38.700 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T16:14:38.700 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T16:14:38.700 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T16:14:38.700 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-11-27T16:14:38.701 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-11-27T16:14:38.701 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-11-27T16:14:38.701 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T16:14:38.702 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T16:14:38.702 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T16:14:38.702 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-11-27T16:14:38.702 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-11-27T16:14:38.703 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-11-27T16:14:38.703 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-11-27T16:14:38.703 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-11-27T16:14:38.703 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-11-27T16:14:38.704 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T16:14:38.704 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T16:14:38.704 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T16:14:38.704 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T16:14:38.705 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T16:14:38.705 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T16:14:38.705 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T16:14:38.705 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-11-27T16:14:38.706 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2022-11-27T16:14:38.706 INFO:teuthology.orchestra.run.smithi134.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-11-27T16:14:38.706 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:14:38.706 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:14:38.706 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-11-27T16:14:38.707 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-11-27T16:14:38.707 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2022-11-27T16:14:38.707 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-11-27T16:14:38.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T16:14:38.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T16:14:38.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T16:14:38.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T16:14:38.709 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T16:14:38.709 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T16:14:38.709 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T16:14:38.710 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T16:14:38.710 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-11-27T16:14:38.710 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:14:38.710 INFO:teuthology.orchestra.run.smithi134.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-11-27T16:14:38.711 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-11-27T16:14:38.711 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-11-27T16:14:38.711 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T16:14:38.711 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-11-27T16:14:38.712 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-11-27T16:14:38.712 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:14:38.712 INFO:teuthology.orchestra.run.smithi134.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-11-27T16:14:38.713 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-11-27T16:14:38.713 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-11-27T16:14:38.713 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T16:14:38.714 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-11-27T16:14:38.714 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T16:14:38.714 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-11-27T16:14:38.715 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-11-27T16:14:38.715 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:14:38.715 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-11-27T16:14:38.716 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-11-27T16:14:38.716 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-11-27T16:14:38.716 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-11-27T16:14:38.716 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-11-27T16:14:38.717 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-11-27T16:14:38.717 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-11-27T16:14:38.718 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-11-27T16:14:38.718 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T16:14:38.718 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-11-27T16:14:38.719 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T16:14:38.719 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T16:14:38.719 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T16:14:38.719 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T16:14:38.720 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T16:14:38.720 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T16:14:38.720 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-11-27T16:14:38.720 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T16:14:38.721 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T16:14:38.721 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-11-27T16:14:38.721 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T16:14:38.722 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T16:14:38.722 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T16:14:38.722 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-11-27T16:14:38.722 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-11-27T16:14:38.723 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-11-27T16:14:38.723 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:14:38.724 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T16:14:38.724 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T16:14:38.724 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-11-27T16:14:38.725 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-11-27T16:14:38.725 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T16:14:38.725 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T16:14:38.726 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-11-27T16:14:38.726 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-11-27T16:14:38.726 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-11-27T16:14:38.726 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T16:14:38.727 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T16:14:38.727 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-11-27T16:14:38.727 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-11-27T16:14:38.728 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-11-27T16:14:38.728 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-11-27T16:14:38.728 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-11-27T16:14:38.728 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-11-27T16:14:38.729 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T16:14:38.729 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:14:38.729 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T16:14:38.729 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T16:14:38.731 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-11-27T16:14:38.731 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T16:14:38.731 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T16:14:38.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T16:14:38.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T16:14:38.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T16:14:38.732 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T16:14:38.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T16:14:38.733 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T16:14:38.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-11-27T16:14:38.734 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T16:14:38.734 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-11-27T16:14:38.734 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T16:14:38.734 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-11-27T16:14:38.735 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-11-27T16:14:38.735 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-11-27T16:14:38.735 INFO:teuthology.orchestra.run.smithi134.stdout:Enabling module streams: 2022-11-27T16:14:38.736 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql 10 2022-11-27T16:14:38.736 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:14:38.736 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2022-11-27T16:14:38.736 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2022-11-27T16:14:38.737 INFO:teuthology.orchestra.run.smithi134.stdout:Install 128 Packages 2022-11-27T16:14:38.737 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 5 Packages 2022-11-27T16:14:38.737 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:14:38.738 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 219 M 2022-11-27T16:14:38.738 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2022-11-27T16:14:39.076 INFO:teuthology.orchestra.run.smithi134.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 18 kB/s | 6.4 kB 00:00 2022-11-27T16:14:39.625 INFO:teuthology.orchestra.run.smithi191.stdout:(8/133): ceph-common-18.0.0-594.g697b0d99.el8.x 9.2 MB/s | 24 MB 00:02 2022-11-27T16:14:39.709 INFO:teuthology.orchestra.run.smithi191.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 293 kB/s | 24 kB 00:00 2022-11-27T16:14:39.739 INFO:teuthology.orchestra.run.smithi134.stdout:(2/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 1.2 MB/s | 845 kB 00:00 2022-11-27T16:14:39.923 INFO:teuthology.orchestra.run.smithi134.stdout:(3/133): ceph-immutable-object-cache-18.0.0-594 903 kB/s | 165 kB 00:00 2022-11-27T16:14:40.409 INFO:teuthology.orchestra.run.smithi191.stdout:(10/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 8.6 MB/s | 18 MB 00:02 2022-11-27T16:14:40.510 INFO:teuthology.orchestra.run.smithi191.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 2.5 MB/s | 257 kB 00:00 2022-11-27T16:14:40.693 INFO:teuthology.orchestra.run.smithi191.stdout:(12/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86 6.5 MB/s | 17 MB 00:02 2022-11-27T16:14:40.718 INFO:teuthology.orchestra.run.smithi191.stdout:(13/133): libcephfs-devel-18.0.0-594.g697b0d99. 146 kB/s | 30 kB 00:00 2022-11-27T16:14:40.793 INFO:teuthology.orchestra.run.smithi191.stdout:(14/133): libcephsqlite-18.0.0-594.g697b0d99.el 1.8 MB/s | 137 kB 00:00 2022-11-27T16:14:40.801 INFO:teuthology.orchestra.run.smithi134.stdout:(4/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 2.5 MB/s | 5.2 MB 00:02 2022-11-27T16:14:40.848 INFO:teuthology.orchestra.run.smithi134.stdout:(5/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 2.4 MB/s | 2.3 MB 00:00 2022-11-27T16:14:40.852 INFO:teuthology.orchestra.run.smithi191.stdout:(15/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 4.7 MB/s | 767 kB 00:00 2022-11-27T16:14:40.877 INFO:teuthology.orchestra.run.smithi191.stdout:(16/133): librados-devel-18.0.0-594.g697b0d99.e 1.6 MB/s | 139 kB 00:00 2022-11-27T16:14:40.952 INFO:teuthology.orchestra.run.smithi191.stdout:(17/133): libradosstriper1-18.0.0-594.g697b0d99 4.6 MB/s | 474 kB 00:00 2022-11-27T16:14:41.045 INFO:teuthology.orchestra.run.smithi191.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 486 kB/s | 45 kB 00:00 2022-11-27T16:14:41.128 INFO:teuthology.orchestra.run.smithi191.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 1.3 MB/s | 109 kB 00:00 2022-11-27T16:14:41.220 INFO:teuthology.orchestra.run.smithi191.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 2.0 MB/s | 190 kB 00:00 2022-11-27T16:14:41.320 INFO:teuthology.orchestra.run.smithi191.stdout:(21/133): python3-rados-18.0.0-594.g697b0d99.el 3.6 MB/s | 371 kB 00:00 2022-11-27T16:14:41.371 INFO:teuthology.orchestra.run.smithi134.stdout:(6/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 2.7 MB/s | 1.5 MB 00:00 2022-11-27T16:14:41.421 INFO:teuthology.orchestra.run.smithi191.stdout:(22/133): python3-rbd-18.0.0-594.g697b0d99.el8. 3.5 MB/s | 357 kB 00:00 2022-11-27T16:14:41.513 INFO:teuthology.orchestra.run.smithi191.stdout:(23/133): python3-rgw-18.0.0-594.g697b0d99.el8. 1.1 MB/s | 107 kB 00:00 2022-11-27T16:14:41.613 INFO:teuthology.orchestra.run.smithi191.stdout:(24/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 946 kB/s | 94 kB 00:00 2022-11-27T16:14:41.882 INFO:teuthology.orchestra.run.smithi134.stdout:(7/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 3.7 MB/s | 3.9 MB 00:01 2022-11-27T16:14:42.347 INFO:teuthology.orchestra.run.smithi191.stdout:(25/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 4.7 MB/s | 3.5 MB 00:00 2022-11-27T16:14:42.439 INFO:teuthology.orchestra.run.smithi191.stdout:(26/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 4.1 MB/s | 6.5 MB 00:01 2022-11-27T16:14:42.464 INFO:teuthology.orchestra.run.smithi191.stdout:(27/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 1.5 MB/s | 175 kB 00:00 2022-11-27T16:14:42.514 INFO:teuthology.orchestra.run.smithi191.stdout:(28/133): ceph-grafana-dashboards-18.0.0-594.g6 305 kB/s | 23 kB 00:00 2022-11-27T16:14:42.540 INFO:teuthology.orchestra.run.smithi191.stdout:(29/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 1.5 MB/s | 118 kB 00:00 2022-11-27T16:14:42.973 INFO:teuthology.orchestra.run.smithi191.stdout:(30/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 3.6 MB/s | 1.7 MB 00:00 2022-11-27T16:14:43.082 INFO:teuthology.orchestra.run.smithi191.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 2.1 MB/s | 234 kB 00:00 2022-11-27T16:14:43.282 INFO:teuthology.orchestra.run.smithi191.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 234 kB/s | 47 kB 00:00 2022-11-27T16:14:43.366 INFO:teuthology.orchestra.run.smithi191.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 173 kB/s | 14 kB 00:00 2022-11-27T16:14:43.440 INFO:teuthology.orchestra.run.smithi134.stdout:(8/133): ceph-common-18.0.0-594.g697b0d99.el8.x 5.1 MB/s | 24 MB 00:04 2022-11-27T16:14:43.441 INFO:teuthology.orchestra.run.smithi191.stdout:(34/133): ceph-volume-18.0.0-594.g697b0d99.el8. 3.4 MB/s | 263 kB 00:00 2022-11-27T16:14:43.499 INFO:teuthology.orchestra.run.smithi134.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 418 kB/s | 24 kB 00:00 2022-11-27T16:14:43.525 INFO:teuthology.orchestra.run.smithi191.stdout:(35/133): cephadm-18.0.0-594.g697b0d99.el8.noar 973 kB/s | 81 kB 00:00 2022-11-27T16:14:43.609 INFO:teuthology.orchestra.run.smithi191.stdout:(36/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 186 kB/s | 15 kB 00:00 2022-11-27T16:14:43.826 INFO:teuthology.orchestra.run.smithi191.stdout:(37/133): ceph-mgr-diskprediction-local-18.0.0- 5.8 MB/s | 7.4 MB 00:01 2022-11-27T16:14:43.851 INFO:teuthology.orchestra.run.smithi191.stdout:(38/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2022-11-27T16:14:43.901 INFO:teuthology.orchestra.run.smithi191.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2022-11-27T16:14:43.960 INFO:teuthology.orchestra.run.smithi191.stdout:(40/133): liboath-2.6.2-3.el8.x86_64.rpm 440 kB/s | 59 kB 00:00 2022-11-27T16:14:44.018 INFO:teuthology.orchestra.run.smithi191.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 748 kB/s | 44 kB 00:00 2022-11-27T16:14:44.060 INFO:teuthology.orchestra.run.smithi191.stdout:(42/133): python3-asyncssh-2.7.0-2.el8.noarch.r 2.9 MB/s | 476 kB 00:00 2022-11-27T16:14:44.111 INFO:teuthology.orchestra.run.smithi191.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 659 kB/s | 33 kB 00:00 2022-11-27T16:14:44.139 INFO:teuthology.orchestra.run.smithi191.stdout:(44/133): python3-beautifulsoup4-4.6.3-2.el8.1. 1.5 MB/s | 185 kB 00:00 2022-11-27T16:14:44.161 INFO:teuthology.orchestra.run.smithi191.stdout:(45/133): python3-certifi-2018.10.15-7.el8.noar 323 kB/s | 16 kB 00:00 2022-11-27T16:14:44.220 INFO:teuthology.orchestra.run.smithi191.stdout:(46/133): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2022-11-27T16:14:44.270 INFO:teuthology.orchestra.run.smithi191.stdout:(47/133): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2022-11-27T16:14:44.295 INFO:teuthology.orchestra.run.smithi191.stdout:(48/133): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2022-11-27T16:14:44.346 INFO:teuthology.orchestra.run.smithi191.stdout:(49/133): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2022-11-27T16:14:44.363 INFO:teuthology.orchestra.run.smithi191.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 261 kB/s | 18 kB 00:00 2022-11-27T16:14:44.405 INFO:teuthology.orchestra.run.smithi191.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2022-11-27T16:14:44.447 INFO:teuthology.orchestra.run.smithi191.stdout:(52/133): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2022-11-27T16:14:44.489 INFO:teuthology.orchestra.run.smithi191.stdout:(53/133): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2022-11-27T16:14:44.597 INFO:teuthology.orchestra.run.smithi191.stdout:(54/133): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2022-11-27T16:14:44.639 INFO:teuthology.orchestra.run.smithi191.stdout:(55/133): python3-portend-2.6-1.el8.noarch.rpm 373 kB/s | 16 kB 00:00 2022-11-27T16:14:44.681 INFO:teuthology.orchestra.run.smithi191.stdout:(56/133): python3-repoze-lru-0.7-6.el8.noarch.r 809 kB/s | 34 kB 00:00 2022-11-27T16:14:44.756 INFO:teuthology.orchestra.run.smithi191.stdout:(57/133): python3-routes-2.4.1-12.el8.noarch.rp 2.6 MB/s | 196 kB 00:00 2022-11-27T16:14:44.790 INFO:teuthology.orchestra.run.smithi191.stdout:(58/133): python3-kubernetes-11.0.0-6.el8.noarc 2.6 MB/s | 1.2 MB 00:00 2022-11-27T16:14:44.816 INFO:teuthology.orchestra.run.smithi191.stdout:(59/133): python3-rsa-4.8-1.el8.noarch.rpm 1.0 MB/s | 61 kB 00:00 2022-11-27T16:14:44.841 INFO:teuthology.orchestra.run.smithi191.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2022-11-27T16:14:44.866 INFO:teuthology.orchestra.run.smithi191.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2022-11-27T16:14:44.891 INFO:teuthology.orchestra.run.smithi191.stdout:(62/133): python3-tempora-1.14.1-5.el8.noarch.r 581 kB/s | 29 kB 00:00 2022-11-27T16:14:44.917 INFO:teuthology.orchestra.run.smithi191.stdout:(63/133): python3-trustme-0.6.0-4.el8.noarch.rp 545 kB/s | 27 kB 00:00 2022-11-27T16:14:44.975 INFO:teuthology.orchestra.run.smithi191.stdout:(64/133): python3-waitress-1.2.1-2.el8.1.noarch 2.8 MB/s | 241 kB 00:00 2022-11-27T16:14:45.001 INFO:teuthology.orchestra.run.smithi191.stdout:(65/133): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2022-11-27T16:14:45.026 INFO:teuthology.orchestra.run.smithi191.stdout:(66/133): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2022-11-27T16:14:45.041 INFO:teuthology.orchestra.run.smithi134.stdout:(10/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 5.8 MB/s | 18 MB 00:03 2022-11-27T16:14:45.043 INFO:teuthology.orchestra.run.smithi191.stdout:(67/133): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2022-11-27T16:14:45.077 INFO:teuthology.orchestra.run.smithi191.stdout:(68/133): python3-zc-lockfile-2.0-2.el8.noarch. 456 kB/s | 23 kB 00:00 2022-11-27T16:14:45.102 INFO:teuthology.orchestra.run.smithi191.stdout:(69/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2022-11-27T16:14:45.116 INFO:teuthology.orchestra.run.smithi134.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 3.3 MB/s | 257 kB 00:00 2022-11-27T16:14:45.161 INFO:teuthology.orchestra.run.smithi191.stdout:(70/133): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2022-11-27T16:14:45.183 INFO:teuthology.orchestra.run.smithi134.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 458 kB/s | 30 kB 00:00 2022-11-27T16:14:45.202 INFO:teuthology.orchestra.run.smithi191.stdout:(71/133): fuse-2.9.7-15.el8.x86_64.rpm 2.0 MB/s | 83 kB 00:00 2022-11-27T16:14:45.245 INFO:teuthology.orchestra.run.smithi191.stdout:(72/133): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2022-11-27T16:14:45.295 INFO:teuthology.orchestra.run.smithi191.stdout:(73/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2022-11-27T16:14:45.317 INFO:teuthology.orchestra.run.smithi134.stdout:(13/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 5.6 MB/s | 767 kB 00:00 2022-11-27T16:14:45.336 INFO:teuthology.orchestra.run.smithi191.stdout:(74/133): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2022-11-27T16:14:45.362 INFO:teuthology.orchestra.run.smithi191.stdout:(75/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 716 kB/s | 18 kB 00:00 2022-11-27T16:14:45.375 INFO:teuthology.orchestra.run.smithi134.stdout:(14/133): libcephsqlite-18.0.0-594.g697b0d99.el 2.3 MB/s | 137 kB 00:00 2022-11-27T16:14:45.387 INFO:teuthology.orchestra.run.smithi191.stdout:(76/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2022-11-27T16:14:45.429 INFO:teuthology.orchestra.run.smithi191.stdout:(77/133): python3-cryptography-3.2.1-5.el8.x86_ 13 MB/s | 559 kB 00:00 2022-11-27T16:14:45.496 INFO:teuthology.orchestra.run.smithi191.stdout:(78/133): python3-oauthlib-2.1.0-1.el8.noarch.r 2.3 MB/s | 155 kB 00:00 2022-11-27T16:14:45.538 INFO:teuthology.orchestra.run.smithi191.stdout:(79/133): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2022-11-27T16:14:45.559 INFO:teuthology.orchestra.run.smithi134.stdout:(15/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86 4.0 MB/s | 17 MB 00:04 2022-11-27T16:14:45.572 INFO:teuthology.orchestra.run.smithi191.stdout:(80/133): thrift-0.13.0-2.el8.x86_64.rpm 3.3 MB/s | 1.7 MB 00:00 2022-11-27T16:14:45.584 INFO:teuthology.orchestra.run.smithi134.stdout:(16/133): librados-devel-18.0.0-594.g697b0d99.e 665 kB/s | 139 kB 00:00 2022-11-27T16:14:45.614 INFO:teuthology.orchestra.run.smithi191.stdout:(81/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2022-11-27T16:14:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:(82/133): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2022-11-27T16:14:45.676 INFO:teuthology.orchestra.run.smithi134.stdout:(17/133): libradosstriper1-18.0.0-594.g697b0d99 4.0 MB/s | 474 kB 00:00 2022-11-27T16:14:45.681 INFO:teuthology.orchestra.run.smithi191.stdout:(83/133): python3-jwt-1.6.1-2.el8.noarch.rpm 635 kB/s | 43 kB 00:00 2022-11-27T16:14:45.707 INFO:teuthology.orchestra.run.smithi191.stdout:(84/133): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 776 kB/s | 58 kB 00:00 2022-11-27T16:14:45.735 INFO:teuthology.orchestra.run.smithi134.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 765 kB/s | 45 kB 00:00 2022-11-27T16:14:45.757 INFO:teuthology.orchestra.run.smithi191.stdout:(85/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.7 MB/s | 238 kB 00:00 2022-11-27T16:14:45.782 INFO:teuthology.orchestra.run.smithi191.stdout:(86/133): elfutils-libelf-devel-0.186-1.el8.x86 597 kB/s | 60 kB 00:00 2022-11-27T16:14:45.810 INFO:teuthology.orchestra.run.smithi134.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 1.4 MB/s | 109 kB 00:00 2022-11-27T16:14:45.824 INFO:teuthology.orchestra.run.smithi191.stdout:(87/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 280 kB/s | 19 kB 00:00 2022-11-27T16:14:45.850 INFO:teuthology.orchestra.run.smithi191.stdout:(88/133): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.4 MB/s | 98 kB 00:00 2022-11-27T16:14:45.885 INFO:teuthology.orchestra.run.smithi134.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 2.5 MB/s | 190 kB 00:00 2022-11-27T16:14:45.891 INFO:teuthology.orchestra.run.smithi191.stdout:(89/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.3 MB/s | 229 kB 00:00 2022-11-27T16:14:45.917 INFO:teuthology.orchestra.run.smithi191.stdout:(90/133): python3-requests-oauthlib-1.0.0-1.el8 640 kB/s | 43 kB 00:00 2022-11-27T16:14:45.959 INFO:teuthology.orchestra.run.smithi191.stdout:(91/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.1 MB/s | 72 kB 00:00 2022-11-27T16:14:45.984 INFO:teuthology.orchestra.run.smithi191.stdout:(92/133): libcom_err-devel-1.45.6-4.el8.x86_64. 578 kB/s | 39 kB 00:00 2022-11-27T16:14:46.019 INFO:teuthology.orchestra.run.smithi134.stdout:(21/133): python3-rados-18.0.0-594.g697b0d99.el 2.7 MB/s | 371 kB 00:00 2022-11-27T16:14:46.026 INFO:teuthology.orchestra.run.smithi191.stdout:(93/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2022-11-27T16:14:46.085 INFO:teuthology.orchestra.run.smithi191.stdout:(94/133): openssl-devel-1.1.1k-7.el8_6.x86_64.r 19 MB/s | 2.3 MB 00:00 2022-11-27T16:14:46.118 INFO:teuthology.orchestra.run.smithi191.stdout:(95/133): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.8 MB/s | 644 kB 00:00 2022-11-27T16:14:46.136 INFO:teuthology.orchestra.run.smithi134.stdout:(22/133): python3-rbd-18.0.0-594.g697b0d99.el8. 3.0 MB/s | 357 kB 00:00 2022-11-27T16:14:46.152 INFO:teuthology.orchestra.run.smithi191.stdout:(96/133): NetworkManager-initscripts-updown-1.3 2.0 MB/s | 138 kB 00:00 2022-11-27T16:14:46.194 INFO:teuthology.orchestra.run.smithi134.stdout:(23/133): python3-rgw-18.0.0-594.g697b0d99.el8. 1.8 MB/s | 107 kB 00:00 2022-11-27T16:14:46.261 INFO:teuthology.orchestra.run.smithi134.stdout:(24/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 1.4 MB/s | 94 kB 00:00 2022-11-27T16:14:46.403 INFO:teuthology.orchestra.run.smithi134.stdout:(25/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 7.9 MB/s | 6.5 MB 00:00 2022-11-27T16:14:46.470 INFO:teuthology.orchestra.run.smithi134.stdout:(26/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 2.6 MB/s | 175 kB 00:00 2022-11-27T16:14:46.554 INFO:teuthology.orchestra.run.smithi134.stdout:(27/133): ceph-grafana-dashboards-18.0.0-594.g6 274 kB/s | 23 kB 00:00 2022-11-27T16:14:46.621 INFO:teuthology.orchestra.run.smithi134.stdout:(28/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 1.7 MB/s | 118 kB 00:00 2022-11-27T16:14:46.661 INFO:teuthology.orchestra.run.smithi191.stdout:(97/133): ceph-test-18.0.0-594.g697b0d99.el8.x8 7.9 MB/s | 55 MB 00:06 2022-11-27T16:14:46.694 INFO:teuthology.orchestra.run.smithi191.stdout:(98/133): postgresql-contrib-10.21-2.module+el8 1.4 MB/s | 808 kB 00:00 2022-11-27T16:14:46.736 INFO:teuthology.orchestra.run.smithi191.stdout:(99/133): bison-3.0.4-10.el8.x86_64.rpm 1.2 MB/s | 688 kB 00:00 2022-11-27T16:14:46.778 INFO:teuthology.orchestra.run.smithi191.stdout:(100/133): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2022-11-27T16:14:46.820 INFO:teuthology.orchestra.run.smithi191.stdout:(101/133): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2022-11-27T16:14:46.846 INFO:teuthology.orchestra.run.smithi134.stdout:(29/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 7.3 MB/s | 1.7 MB 00:00 2022-11-27T16:14:46.879 INFO:teuthology.orchestra.run.smithi191.stdout:(102/133): python3-numpy-1.14.3-10.el8.x86_64.r 20 MB/s | 3.7 MB 00:00 2022-11-27T16:14:46.896 INFO:teuthology.orchestra.run.smithi134.stdout:(30/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 5.5 MB/s | 3.5 MB 00:00 2022-11-27T16:14:46.913 INFO:teuthology.orchestra.run.smithi191.stdout:(103/133): python3-markupsafe-0.23-19.el8.x86_6 423 kB/s | 39 kB 00:00 2022-11-27T16:14:46.938 INFO:teuthology.orchestra.run.smithi191.stdout:(104/133): python3-prettytable-0.7.2-14.el8.noa 761 kB/s | 44 kB 00:00 2022-11-27T16:14:46.972 INFO:teuthology.orchestra.run.smithi134.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 3.0 MB/s | 234 kB 00:00 2022-11-27T16:14:47.013 INFO:teuthology.orchestra.run.smithi191.stdout:(105/133): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2022-11-27T16:14:47.039 INFO:teuthology.orchestra.run.smithi134.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 705 kB/s | 47 kB 00:00 2022-11-27T16:14:47.046 INFO:teuthology.orchestra.run.smithi191.stdout:(106/133): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2022-11-27T16:14:47.080 INFO:teuthology.orchestra.run.smithi191.stdout:(107/133): python3-jinja2-2.10.1-3.el8.noarch.r 3.7 MB/s | 538 kB 00:00 2022-11-27T16:14:47.106 INFO:teuthology.orchestra.run.smithi134.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 216 kB/s | 14 kB 00:00 2022-11-27T16:14:47.107 INFO:teuthology.orchestra.run.smithi191.stdout:(108/133): libtool-2.4.6-25.el8.x86_64.rpm 7.7 MB/s | 709 kB 00:00 2022-11-27T16:14:47.140 INFO:teuthology.orchestra.run.smithi191.stdout:(109/133): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2022-11-27T16:14:47.156 INFO:teuthology.orchestra.run.smithi191.stdout:(110/133): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2022-11-27T16:14:47.199 INFO:teuthology.orchestra.run.smithi191.stdout:(111/133): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.5 MB/s | 103 kB 00:00 2022-11-27T16:14:47.206 INFO:teuthology.orchestra.run.smithi134.stdout:(34/133): ceph-volume-18.0.0-594.g697b0d99.el8. 2.6 MB/s | 263 kB 00:00 2022-11-27T16:14:47.240 INFO:teuthology.orchestra.run.smithi191.stdout:(112/133): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2022-11-27T16:14:47.281 INFO:teuthology.orchestra.run.smithi134.stdout:(35/133): cephadm-18.0.0-594.g697b0d99.el8.noar 1.1 MB/s | 81 kB 00:00 2022-11-27T16:14:47.307 INFO:teuthology.orchestra.run.smithi191.stdout:(113/133): openblas-threads-0.3.15-3.el8.x86_64 21 MB/s | 4.9 MB 00:00 2022-11-27T16:14:47.348 INFO:teuthology.orchestra.run.smithi134.stdout:(36/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 232 kB/s | 15 kB 00:00 2022-11-27T16:14:47.382 INFO:teuthology.orchestra.run.smithi191.stdout:(114/133): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2022-11-27T16:14:47.407 INFO:teuthology.orchestra.run.smithi191.stdout:(115/133): flex-2.6.1-9.el8.x86_64.rpm 1.9 MB/s | 320 kB 00:00 2022-11-27T16:14:47.441 INFO:teuthology.orchestra.run.smithi191.stdout:(116/133): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2022-11-27T16:14:47.500 INFO:teuthology.orchestra.run.smithi191.stdout:(117/133): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2022-11-27T16:14:47.574 INFO:teuthology.orchestra.run.smithi134.stdout:(37/133): ceph-mgr-diskprediction-local-18.0.0- 10 MB/s | 7.4 MB 00:00 2022-11-27T16:14:47.575 INFO:teuthology.orchestra.run.smithi191.stdout:(118/133): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2022-11-27T16:14:47.609 INFO:teuthology.orchestra.run.smithi191.stdout:(119/133): python3-pytz-2017.2-9.el8.noarch.rpm 500 kB/s | 54 kB 00:00 2022-11-27T16:14:47.624 INFO:teuthology.orchestra.run.smithi134.stdout:(38/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2022-11-27T16:14:47.634 INFO:teuthology.orchestra.run.smithi191.stdout:(120/133): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.2 MB/s | 126 kB 00:00 2022-11-27T16:14:47.666 INFO:teuthology.orchestra.run.smithi134.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2022-11-27T16:14:47.667 INFO:teuthology.orchestra.run.smithi191.stdout:(121/133): python3-jsonpointer-1.10-11.el8.noar 342 kB/s | 20 kB 00:00 2022-11-27T16:14:47.693 INFO:teuthology.orchestra.run.smithi191.stdout:(122/133): libpq-13.5-1.el8.x86_64.rpm 3.4 MB/s | 198 kB 00:00 2022-11-27T16:14:47.743 INFO:teuthology.orchestra.run.smithi191.stdout:(123/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2022-11-27T16:14:47.768 INFO:teuthology.orchestra.run.smithi191.stdout:(124/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2022-11-27T16:14:47.811 INFO:teuthology.orchestra.run.smithi191.stdout:(125/133): python3-jsonpatch-1.21-2.el8.noarch. 397 kB/s | 27 kB 00:00 2022-11-27T16:14:47.836 INFO:teuthology.orchestra.run.smithi191.stdout:(126/133): uuid-1.6.2-43.el8.x86_64.rpm 973 kB/s | 64 kB 00:00 2022-11-27T16:14:47.841 INFO:teuthology.orchestra.run.smithi134.stdout:(40/133): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2022-11-27T16:14:47.870 INFO:teuthology.orchestra.run.smithi191.stdout:(127/133): python3-pyasn1-modules-0.3.7-6.el8.n 1.8 MB/s | 110 kB 00:00 2022-11-27T16:14:47.891 INFO:teuthology.orchestra.run.smithi134.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 878 kB/s | 44 kB 00:00 2022-11-27T16:14:47.917 INFO:teuthology.orchestra.run.smithi134.stdout:(42/133): liboath-2.6.2-3.el8.x86_64.rpm 171 kB/s | 59 kB 00:00 2022-11-27T16:14:47.967 INFO:teuthology.orchestra.run.smithi134.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 660 kB/s | 33 kB 00:00 2022-11-27T16:14:47.992 INFO:teuthology.orchestra.run.smithi134.stdout:(44/133): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2022-11-27T16:14:48.017 INFO:teuthology.orchestra.run.smithi134.stdout:(45/133): python3-certifi-2018.10.15-7.el8.noar 326 kB/s | 16 kB 00:00 2022-11-27T16:14:48.036 INFO:teuthology.orchestra.run.smithi191.stdout:(128/133): python3-scipy-1.0.0-21.module+el8.5. 22 MB/s | 14 MB 00:00 2022-11-27T16:14:48.068 INFO:teuthology.orchestra.run.smithi134.stdout:(46/133): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2022-11-27T16:14:48.079 INFO:teuthology.orchestra.run.smithi191.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 17 MB/s | 709 kB 00:00 2022-11-27T16:14:48.118 INFO:teuthology.orchestra.run.smithi134.stdout:(47/133): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2022-11-27T16:14:48.137 INFO:teuthology.orchestra.run.smithi191.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 1.5 MB 00:00 2022-11-27T16:14:48.169 INFO:teuthology.orchestra.run.smithi134.stdout:(48/133): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2022-11-27T16:14:48.187 INFO:teuthology.orchestra.run.smithi191.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2022-11-27T16:14:48.194 INFO:teuthology.orchestra.run.smithi134.stdout:(49/133): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2022-11-27T16:14:48.219 INFO:teuthology.orchestra.run.smithi134.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 351 kB/s | 18 kB 00:00 2022-11-27T16:14:48.294 INFO:teuthology.orchestra.run.smithi134.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 649 kB/s | 49 kB 00:00 2022-11-27T16:14:48.345 INFO:teuthology.orchestra.run.smithi134.stdout:(52/133): python3-more-itertools-7.2.0-3.el8.no 1.2 MB/s | 59 kB 00:00 2022-11-27T16:14:48.395 INFO:teuthology.orchestra.run.smithi134.stdout:(53/133): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2022-11-27T16:14:48.512 INFO:teuthology.orchestra.run.smithi134.stdout:(54/133): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2022-11-27T16:14:48.529 INFO:teuthology.orchestra.run.smithi191.stdout:(132/133): librados2-18.0.0-594.g697b0d99.el8.x 5.2 MB/s | 3.6 MB 00:00 2022-11-27T16:14:48.545 INFO:teuthology.orchestra.run.smithi134.stdout:(55/133): python3-kubernetes-11.0.0-6.el8.noarc 3.3 MB/s | 1.2 MB 00:00 2022-11-27T16:14:48.571 INFO:teuthology.orchestra.run.smithi134.stdout:(56/133): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2022-11-27T16:14:48.596 INFO:teuthology.orchestra.run.smithi134.stdout:(57/133): python3-repoze-lru-0.7-6.el8.noarch.r 672 kB/s | 34 kB 00:00 2022-11-27T16:14:48.646 INFO:teuthology.orchestra.run.smithi191.stdout:(133/133): librbd1-18.0.0-594.g697b0d99.el8.x86 4.6 MB/s | 3.6 MB 00:00 2022-11-27T16:14:48.647 INFO:teuthology.orchestra.run.smithi134.stdout:(58/133): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2022-11-27T16:14:48.649 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:14:48.650 INFO:teuthology.orchestra.run.smithi191.stdout:Total 19 MB/s | 219 MB 00:11 2022-11-27T16:14:48.672 INFO:teuthology.orchestra.run.smithi134.stdout:(59/133): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2022-11-27T16:14:48.697 INFO:teuthology.orchestra.run.smithi134.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2022-11-27T16:14:48.722 INFO:teuthology.orchestra.run.smithi134.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 476 kB/s | 24 kB 00:00 2022-11-27T16:14:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:(62/133): ceph-test-18.0.0-594.g697b0d99.el8.x8 9.6 MB/s | 55 MB 00:05 2022-11-27T16:14:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:(63/133): python3-tempora-1.14.1-5.el8.noarch.r 55 kB/s | 29 kB 00:00 2022-11-27T16:14:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:(64/133): python3-trustme-0.6.0-4.el8.noarch.rp 52 kB/s | 27 kB 00:00 2022-11-27T16:14:49.340 INFO:teuthology.orchestra.run.smithi134.stdout:(65/133): python3-websocket-client-0.56.0-5.el8 738 kB/s | 61 kB 00:00 2022-11-27T16:14:49.390 INFO:teuthology.orchestra.run.smithi134.stdout:(66/133): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2022-11-27T16:14:49.432 INFO:teuthology.orchestra.run.smithi134.stdout:(67/133): python3-waitress-1.2.1-2.el8.1.noarch 1.0 MB/s | 241 kB 00:00 2022-11-27T16:14:49.458 INFO:teuthology.orchestra.run.smithi134.stdout:(68/133): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2022-11-27T16:14:49.491 INFO:teuthology.orchestra.run.smithi134.stdout:(69/133): python3-zc-lockfile-2.0-2.el8.noarch. 228 kB/s | 23 kB 00:00 2022-11-27T16:14:49.514 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2022-11-27T16:14:49.516 INFO:teuthology.orchestra.run.smithi134.stdout:(70/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2022-11-27T16:14:49.558 INFO:teuthology.orchestra.run.smithi134.stdout:(71/133): postgresql-contrib-10.21-2.module+el8 12 MB/s | 808 kB 00:00 2022-11-27T16:14:49.600 INFO:teuthology.orchestra.run.smithi134.stdout:(72/133): bison-3.0.4-10.el8.x86_64.rpm 8.1 MB/s | 688 kB 00:00 2022-11-27T16:14:49.651 INFO:teuthology.orchestra.run.smithi134.stdout:(73/133): postgresql-10.21-2.module+el8.6.0+153 16 MB/s | 1.5 MB 00:00 2022-11-27T16:14:49.673 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2022-11-27T16:14:49.674 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2022-11-27T16:14:49.726 INFO:teuthology.orchestra.run.smithi134.stdout:(74/133): python3-numpy-1.14.3-10.el8.x86_64.rp 30 MB/s | 3.7 MB 00:00 2022-11-27T16:14:49.768 INFO:teuthology.orchestra.run.smithi134.stdout:(75/133): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2022-11-27T16:14:49.835 INFO:teuthology.orchestra.run.smithi134.stdout:(76/133): postgresql-server-10.21-2.module+el8. 28 MB/s | 5.1 MB 00:00 2022-11-27T16:14:49.876 INFO:teuthology.orchestra.run.smithi134.stdout:(77/133): thrift-0.13.0-2.el8.x86_64.rpm 3.9 MB/s | 1.7 MB 00:00 2022-11-27T16:14:49.893 INFO:teuthology.orchestra.run.smithi134.stdout:(78/133): python3-markupsafe-0.23-19.el8.x86_64 310 kB/s | 39 kB 00:00 2022-11-27T16:14:49.927 INFO:teuthology.orchestra.run.smithi134.stdout:(79/133): python3-prettytable-0.7.2-14.el8.noar 478 kB/s | 44 kB 00:00 2022-11-27T16:14:49.961 INFO:teuthology.orchestra.run.smithi134.stdout:(80/133): librdkafka-0.11.4-3.el8.x86_64.rpm 5.2 MB/s | 354 kB 00:00 2022-11-27T16:14:50.020 INFO:teuthology.orchestra.run.smithi134.stdout:(81/133): python3-jinja2-2.10.1-3.el8.noarch.rp 4.2 MB/s | 538 kB 00:00 2022-11-27T16:14:50.061 INFO:teuthology.orchestra.run.smithi134.stdout:(82/133): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2022-11-27T16:14:50.087 INFO:teuthology.orchestra.run.smithi134.stdout:(83/133): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.8 MB/s | 225 kB 00:00 2022-11-27T16:14:50.112 INFO:teuthology.orchestra.run.smithi134.stdout:(84/133): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2022-11-27T16:14:50.154 INFO:teuthology.orchestra.run.smithi134.stdout:(85/133): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2022-11-27T16:14:50.229 INFO:teuthology.orchestra.run.smithi134.stdout:(86/133): openblas-threads-0.3.15-3.el8.x86_64. 23 MB/s | 4.9 MB 00:00 2022-11-27T16:14:50.254 INFO:teuthology.orchestra.run.smithi134.stdout:(87/133): socat-1.7.4.1-1.el8.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2022-11-27T16:14:50.288 INFO:teuthology.orchestra.run.smithi134.stdout:(88/133): flex-2.6.1-9.el8.x86_64.rpm 5.4 MB/s | 320 kB 00:00 2022-11-27T16:14:50.346 INFO:teuthology.orchestra.run.smithi134.stdout:(89/133): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2022-11-27T16:14:50.380 INFO:teuthology.orchestra.run.smithi134.stdout:(90/133): jq-1.6-3.el8.x86_64.rpm 2.1 MB/s | 202 kB 00:00 2022-11-27T16:14:50.430 INFO:teuthology.orchestra.run.smithi134.stdout:(91/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-11-27T16:14:50.497 INFO:teuthology.orchestra.run.smithi134.stdout:(92/133): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2022-11-27T16:14:50.539 INFO:teuthology.orchestra.run.smithi134.stdout:(93/133): python3-pytz-2017.2-9.el8.noarch.rpm 500 kB/s | 54 kB 00:00 2022-11-27T16:14:50.564 INFO:teuthology.orchestra.run.smithi134.stdout:(94/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.9 MB/s | 126 kB 00:00 2022-11-27T16:14:50.623 INFO:teuthology.orchestra.run.smithi134.stdout:(95/133): python3-jsonpointer-1.10-11.el8.noarc 239 kB/s | 20 kB 00:00 2022-11-27T16:14:50.648 INFO:teuthology.orchestra.run.smithi134.stdout:(96/133): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2022-11-27T16:14:50.690 INFO:teuthology.orchestra.run.smithi134.stdout:(97/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2022-11-27T16:14:50.716 INFO:teuthology.orchestra.run.smithi134.stdout:(98/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2022-11-27T16:14:50.758 INFO:teuthology.orchestra.run.smithi134.stdout:(99/133): python3-jsonpatch-1.21-2.el8.noarch.r 395 kB/s | 27 kB 00:00 2022-11-27T16:14:50.783 INFO:teuthology.orchestra.run.smithi134.stdout:(100/133): uuid-1.6.2-43.el8.x86_64.rpm 968 kB/s | 64 kB 00:00 2022-11-27T16:14:50.839 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2022-11-27T16:14:50.843 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2022-11-27T16:14:50.933 INFO:teuthology.orchestra.run.smithi134.stdout:(101/133): python3-scipy-1.0.0-21.module+el8.5. 23 MB/s | 14 MB 00:00 2022-11-27T16:14:50.959 INFO:teuthology.orchestra.run.smithi134.stdout:(102/133): python3-pyasn1-modules-0.3.7-6.el8.n 549 kB/s | 110 kB 00:00 2022-11-27T16:14:50.992 INFO:teuthology.orchestra.run.smithi134.stdout:(103/133): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2022-11-27T16:14:51.026 INFO:teuthology.orchestra.run.smithi134.stdout:(104/133): fuse-2.9.7-15.el8.x86_64.rpm 899 kB/s | 83 kB 00:00 2022-11-27T16:14:51.051 INFO:teuthology.orchestra.run.smithi134.stdout:(105/133): libsepol-devel-2.9-3.el8.x86_64.rpm 956 kB/s | 87 kB 00:00 2022-11-27T16:14:51.076 INFO:teuthology.orchestra.run.smithi134.stdout:(106/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2022-11-27T16:14:51.118 INFO:teuthology.orchestra.run.smithi134.stdout:(107/133): libselinux-devel-2.9-5.el8.x86_64.rp 2.1 MB/s | 200 kB 00:00 2022-11-27T16:14:51.166 INFO:teuthology.orchestra.run.smithi134.stdout:(108/133): libverto-devel-0.3.0-5.el8.x86_64.rp 155 kB/s | 18 kB 00:00 2022-11-27T16:14:51.194 INFO:teuthology.orchestra.run.smithi134.stdout:(109/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 403 kB/s | 47 kB 00:00 2022-11-27T16:14:51.244 INFO:teuthology.orchestra.run.smithi134.stdout:(110/133): python3-cryptography-3.2.1-5.el8.x86 4.4 MB/s | 559 kB 00:00 2022-11-27T16:14:51.270 INFO:teuthology.orchestra.run.smithi134.stdout:(111/133): python3-pycparser-2.14-14.el8.noarch 1.4 MB/s | 109 kB 00:00 2022-11-27T16:14:51.295 INFO:teuthology.orchestra.run.smithi134.stdout:(112/133): python3-oauthlib-2.1.0-1.el8.noarch. 1.2 MB/s | 155 kB 00:00 2022-11-27T16:14:51.337 INFO:teuthology.orchestra.run.smithi134.stdout:(113/133): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2022-11-27T16:14:51.353 INFO:teuthology.orchestra.run.smithi134.stdout:(114/133): python3-jwt-1.6.1-2.el8.noarch.rpm 744 kB/s | 43 kB 00:00 2022-11-27T16:14:51.387 INFO:teuthology.orchestra.run.smithi134.stdout:(115/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.8 MB/s | 560 kB 00:00 2022-11-27T16:14:51.412 INFO:teuthology.orchestra.run.smithi134.stdout:(116/133): zlib-devel-1.2.11-18.el8_5.x86_64.rp 765 kB/s | 58 kB 00:00 2022-11-27T16:14:51.429 INFO:teuthology.orchestra.run.smithi134.stdout:(117/133): elfutils-libelf-devel-0.186-1.el8.x8 794 kB/s | 60 kB 00:00 2022-11-27T16:14:51.446 INFO:teuthology.orchestra.run.smithi134.stdout:(118/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.1 MB/s | 238 kB 00:00 2022-11-27T16:14:51.471 INFO:teuthology.orchestra.run.smithi134.stdout:(119/133): libaio-devel-0.3.112-1.el8.x86_64.rp 318 kB/s | 19 kB 00:00 2022-11-27T16:14:51.497 INFO:teuthology.orchestra.run.smithi134.stdout:(120/133): libuuid-devel-2.32.1-35.el8.x86_64.r 1.4 MB/s | 98 kB 00:00 2022-11-27T16:14:51.530 INFO:teuthology.orchestra.run.smithi134.stdout:(121/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2022-11-27T16:14:51.556 INFO:teuthology.orchestra.run.smithi134.stdout:(122/133): python3-requests-oauthlib-1.0.0-1.el 509 kB/s | 43 kB 00:00 2022-11-27T16:14:51.581 INFO:teuthology.orchestra.run.smithi134.stdout:(123/133): xfsprogs-devel-5.0.0-10.el8.x86_64.r 874 kB/s | 72 kB 00:00 2022-11-27T16:14:51.614 INFO:teuthology.orchestra.run.smithi134.stdout:(124/133): libcom_err-devel-1.45.6-4.el8.x86_64 461 kB/s | 39 kB 00:00 2022-11-27T16:14:51.631 INFO:teuthology.orchestra.run.smithi134.stdout:(125/133): keyutils-libs-devel-1.5.10-9.el8.x86 982 kB/s | 48 kB 00:00 2022-11-27T16:14:51.673 INFO:teuthology.orchestra.run.smithi134.stdout:(126/133): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2022-11-27T16:14:51.707 INFO:teuthology.orchestra.run.smithi134.stdout:(127/133): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.8 MB/s | 644 kB 00:00 2022-11-27T16:14:51.782 INFO:teuthology.orchestra.run.smithi134.stdout:(128/133): openssl-devel-1.1.1k-7.el8_6.x86_64. 10 MB/s | 2.3 MB 00:00 2022-11-27T16:14:51.824 INFO:teuthology.orchestra.run.smithi134.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 17 MB/s | 709 kB 00:00 2022-11-27T16:14:51.883 INFO:teuthology.orchestra.run.smithi134.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 1.5 MB 00:00 2022-11-27T16:14:51.925 INFO:teuthology.orchestra.run.smithi134.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2022-11-27T16:14:52.150 INFO:teuthology.orchestra.run.smithi134.stdout:(132/133): librados2-18.0.0-594.g697b0d99.el8.x 7.6 MB/s | 3.6 MB 00:00 2022-11-27T16:14:52.342 INFO:teuthology.orchestra.run.smithi134.stdout:(133/133): librbd1-18.0.0-594.g697b0d99.el8.x86 5.6 MB/s | 3.6 MB 00:00 2022-11-27T16:14:52.345 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:14:52.345 INFO:teuthology.orchestra.run.smithi134.stdout:Total 16 MB/s | 219 MB 00:13 2022-11-27T16:14:53.221 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2022-11-27T16:14:53.380 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2022-11-27T16:14:53.380 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2022-11-27T16:14:54.550 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2022-11-27T16:14:54.555 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2022-11-27T16:14:56.050 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2022-11-27T16:14:56.289 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:14:56.501 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:14:56.701 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-11-27T16:14:56.842 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-11-27T16:14:57.009 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/138 2022-11-27T16:14:57.162 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/138 2022-11-27T16:14:57.261 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/138 2022-11-27T16:14:57.411 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:14:57.802 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:14:58.028 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:14:58.410 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:14:58.580 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:14:58.696 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:14:58.886 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:14:59.002 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:14:59.183 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:14:59.397 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-11-27T16:14:59.681 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-11-27T16:15:00.024 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2022-11-27T16:15:00.037 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-11-27T16:15:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:15:00.261 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-11-27T16:15:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:15:00.456 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:15:00.524 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:15:00.614 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-11-27T16:15:00.669 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:15:00.747 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:15:00.781 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-11-27T16:15:00.900 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T16:15:00.922 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 17/138 2022-11-27T16:15:00.993 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T16:15:01.135 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/138 2022-11-27T16:15:01.175 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 5/138 2022-11-27T16:15:01.317 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:15:01.356 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/138 2022-11-27T16:15:01.490 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 20/138 2022-11-27T16:15:01.602 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-11-27T16:15:01.715 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:15:01.740 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-11-27T16:15:01.852 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-11-27T16:15:01.925 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:15:02.313 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:15:02.462 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-11-27T16:15:02.484 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:15:02.600 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:15:02.677 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-11-27T16:15:02.850 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:15:02.931 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:15:03.114 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:15:03.255 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-11-27T16:15:03.377 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-11-27T16:15:03.460 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.15-3.el8.x86_64 24/138 2022-11-27T16:15:03.536 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-11-27T16:15:03.950 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-11-27T16:15:04.119 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 24/138 2022-11-27T16:15:04.137 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-11-27T16:15:04.298 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:15:04.394 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:15:04.423 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/138 2022-11-27T16:15:04.549 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:15:04.617 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:15:04.772 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 17/138 2022-11-27T16:15:04.899 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T16:15:05.030 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T16:15:05.068 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-11-27T16:15:05.158 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T16:15:05.217 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/138 2022-11-27T16:15:05.354 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/138 2022-11-27T16:15:05.627 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/138 2022-11-27T16:15:05.759 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T16:15:05.835 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-11-27T16:15:05.932 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T16:15:06.517 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T16:15:06.675 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T16:15:07.373 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T16:15:07.558 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-11-27T16:15:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 22/138 2022-11-27T16:15:07.835 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 23/138 2022-11-27T16:15:07.902 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/138 2022-11-27T16:15:08.125 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/138 2022-11-27T16:15:08.400 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/138 2022-11-27T16:15:08.674 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-11-27T16:15:08.853 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/138 2022-11-27T16:15:09.039 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/138 2022-11-27T16:15:09.168 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/138 2022-11-27T16:15:09.213 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/138 2022-11-27T16:15:09.406 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/138 2022-11-27T16:15:09.451 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/138 2022-11-27T16:15:09.678 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-11-27T16:15:09.868 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-11-27T16:15:10.064 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 37/138 2022-11-27T16:15:10.270 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 38/138 2022-11-27T16:15:10.288 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 30/138 2022-11-27T16:15:10.501 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-11-27T16:15:10.670 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/138 2022-11-27T16:15:10.692 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/138 2022-11-27T16:15:10.783 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/138 2022-11-27T16:15:10.950 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/138 2022-11-27T16:15:10.951 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 41/138 2022-11-27T16:15:11.016 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 41/138 2022-11-27T16:15:11.141 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T16:15:11.198 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.6-3.el8.x86_64 42/138 2022-11-27T16:15:11.383 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 43/138 2022-11-27T16:15:11.627 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 44/138 2022-11-27T16:15:11.858 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.1-7.el8.noarch 45/138 2022-11-27T16:15:12.029 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 46/138 2022-11-27T16:15:12.163 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 47/138 2022-11-27T16:15:12.329 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/138 2022-11-27T16:15:12.550 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 49/138 2022-11-27T16:15:12.740 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 50/138 2022-11-27T16:15:12.858 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T16:15:12.964 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/138 2022-11-27T16:15:13.191 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/138 2022-11-27T16:15:13.361 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/138 2022-11-27T16:15:13.384 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-11-27T16:15:13.566 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-11-27T16:15:13.689 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/138 2022-11-27T16:15:13.804 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 37/138 2022-11-27T16:15:13.886 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/138 2022-11-27T16:15:14.010 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 38/138 2022-11-27T16:15:14.118 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/138 2022-11-27T16:15:14.243 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-11-27T16:15:14.362 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/138 2022-11-27T16:15:14.377 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 40/138 2022-11-27T16:15:14.534 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 41/138 2022-11-27T16:15:14.689 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/138 2022-11-27T16:15:14.722 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/138 2022-11-27T16:15:14.941 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/138 2022-11-27T16:15:14.944 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 43/138 2022-11-27T16:15:15.117 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/138 2022-11-27T16:15:15.159 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 44/138 2022-11-27T16:15:15.352 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/138 2022-11-27T16:15:15.494 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 61/138 2022-11-27T16:15:15.552 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 45/138 2022-11-27T16:15:15.827 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/138 2022-11-27T16:15:15.867 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 46/138 2022-11-27T16:15:16.071 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/138 2022-11-27T16:15:16.090 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 47/138 2022-11-27T16:15:16.314 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 64/138 2022-11-27T16:15:16.334 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 48/138 2022-11-27T16:15:16.570 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/138 2022-11-27T16:15:16.602 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 49/138 2022-11-27T16:15:16.825 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/138 2022-11-27T16:15:16.886 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 50/138 2022-11-27T16:15:17.008 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.7-15.el8.x86_64 67/138 2022-11-27T16:15:17.105 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 51/138 2022-11-27T16:15:17.215 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-11-27T16:15:17.298 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 52/138 2022-11-27T16:15:17.377 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:15:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 53/138 2022-11-27T16:15:17.685 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-11-27T16:15:17.691 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 54/138 2022-11-27T16:15:17.897 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-11-27T16:15:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/138 2022-11-27T16:15:18.062 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-11-27T16:15:18.225 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/138 2022-11-27T16:15:18.259 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-11-27T16:15:18.465 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-11-27T16:15:18.499 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 57/138 2022-11-27T16:15:18.660 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-11-27T16:15:18.767 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 58/138 2022-11-27T16:15:18.858 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-11-27T16:15:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 59/138 2022-11-27T16:15:19.035 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-11-27T16:15:19.196 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-11-27T16:15:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.7-15.el8.x86_64 60/138 2022-11-27T16:15:19.414 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T16:15:19.452 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-11-27T16:15:19.495 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T16:15:19.651 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-11-27T16:15:19.691 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T16:15:19.768 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T16:15:19.822 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-11-27T16:15:19.920 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : jq-1.6-3.el8.x86_64 63/138 2022-11-27T16:15:20.084 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-11-27T16:15:20.121 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 64/138 2022-11-27T16:15:20.281 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-11-27T16:15:20.368 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 65/138 2022-11-27T16:15:20.481 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-11-27T16:15:20.614 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.1-7.el8.noarch 66/138 2022-11-27T16:15:20.706 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-11-27T16:15:20.765 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 67/138 2022-11-27T16:15:20.955 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-11-27T16:15:21.143 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:15:21.406 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-11-27T16:15:21.612 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-11-27T16:15:21.762 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-11-27T16:15:21.818 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-11-27T16:15:21.975 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-11-27T16:15:22.035 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-11-27T16:15:22.173 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-11-27T16:15:22.242 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-11-27T16:15:22.401 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-11-27T16:15:22.513 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-11-27T16:15:22.583 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-11-27T16:15:22.750 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-11-27T16:15:22.841 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-11-27T16:15:22.936 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-11-27T16:15:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-11-27T16:15:23.125 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-11-27T16:15:23.334 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-11-27T16:15:23.519 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-11-27T16:15:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-11-27T16:15:23.784 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-11-27T16:15:23.937 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-11-27T16:15:23.956 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:15:24.017 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:15:24.136 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-11-27T16:15:24.216 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 94/138 2022-11-27T16:15:24.325 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-11-27T16:15:25.368 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-11-27T16:15:25.624 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-11-27T16:15:25.872 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-11-27T16:15:26.145 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-11-27T16:15:26.474 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-11-27T16:15:26.654 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-11-27T16:15:27.416 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-11-27T16:15:27.596 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:15:27.672 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:15:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 94/138 2022-11-27T16:15:28.009 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:15:28.240 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:15:28.794 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:15:29.050 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:15:29.130 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:15:29.264 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:15:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:15:31.871 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:15:32.427 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:15:32.716 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:15:32.798 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:15:32.920 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:15:48.229 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:15:48.230 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2022-11-27T16:15:48.230 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2022-11-27T16:15:48.230 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2022-11-27T16:15:48.231 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /run 2022-11-27T16:15:48.231 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2022-11-27T16:15:48.232 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2022-11-27T16:15:48.232 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2022-11-27T16:15:48.233 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2022-11-27T16:15:48.233 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2022-11-27T16:15:48.233 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:15:48.619 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:15:50.099 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:15:50.280 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:15:50.495 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:15:50.669 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:15:51.049 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:15:51.202 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:15:51.219 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:15:51.399 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:15:51.399 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2022-11-27T16:15:51.400 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2022-11-27T16:15:51.400 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2022-11-27T16:15:51.400 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /run 2022-11-27T16:15:51.401 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2022-11-27T16:15:51.401 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2022-11-27T16:15:51.401 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2022-11-27T16:15:51.401 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2022-11-27T16:15:51.402 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2022-11-27T16:15:51.402 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:15:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:15:52.756 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:15:52.915 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:15:53.057 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 103/138 2022-11-27T16:15:53.229 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 104/138 2022-11-27T16:15:53.258 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:15:53.393 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:15:53.479 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:15:53.597 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:15:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:15:53.851 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:15:53.886 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:15:54.231 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:15:54.384 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:15:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:15:54.844 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:15:54.993 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:15:55.083 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:15:55.454 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 108/138 2022-11-27T16:15:55.631 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:15:55.666 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:15:55.818 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:15:55.829 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:15:55.988 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:15:56.148 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:15:56.289 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 103/138 2022-11-27T16:15:56.461 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 104/138 2022-11-27T16:15:56.633 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:15:56.838 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:15:57.145 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:15:57.534 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 111/138 2022-11-27T16:15:57.742 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:15:57.776 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:15:57.917 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:15:57.959 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:15:58.092 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:15:58.105 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:15:58.257 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:15:58.347 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:15:58.438 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:15:58.617 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:15:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 108/138 2022-11-27T16:15:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:15:58.942 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:15:59.099 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:15:59.111 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:16:00.857 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 111/138 2022-11-27T16:16:01.060 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:16:01.093 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:16:01.249 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:16:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:16:01.466 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:16:01.813 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:16:01.966 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:16:03.700 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:16:03.996 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 116/138 2022-11-27T16:16:04.783 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 117/138 2022-11-27T16:16:05.051 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/138 2022-11-27T16:16:05.309 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/138 2022-11-27T16:16:05.457 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/138 2022-11-27T16:16:05.647 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/138 2022-11-27T16:16:05.887 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/138 2022-11-27T16:16:06.061 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 122/138 2022-11-27T16:16:06.183 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-11-27T16:16:07.049 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:16:07.374 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 116/138 2022-11-27T16:16:07.523 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T16:16:07.710 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T16:16:08.046 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 118/138 2022-11-27T16:16:08.200 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:16:08.474 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:16:08.492 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:16:08.657 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 125/138 2022-11-27T16:16:08.858 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 119/138 2022-11-27T16:16:08.881 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 126/138 2022-11-27T16:16:09.147 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 127/138 2022-11-27T16:16:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 120/138 2022-11-27T16:16:09.319 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 121/138 2022-11-27T16:16:09.487 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-11-27T16:16:09.527 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 122/138 2022-11-27T16:16:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-11-27T16:16:09.825 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-11-27T16:16:10.048 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/138 2022-11-27T16:16:10.122 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/138 2022-11-27T16:16:10.323 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/138 2022-11-27T16:16:10.336 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/138 2022-11-27T16:16:10.498 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-11-27T16:16:10.507 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-11-27T16:16:10.668 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 133/138 2022-11-27T16:16:10.834 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:16:10.834 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:16:11.042 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:16:11.042 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:16:11.285 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:16:11.285 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:16:11.553 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:16:11.732 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:16:11.739 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:16:12.031 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:16:12.049 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:16:12.309 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 125/138 2022-11-27T16:16:12.318 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:16:12.319 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/138 2022-11-27T16:16:12.319 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/138 2022-11-27T16:16:12.319 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/138 2022-11-27T16:16:12.320 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/138 2022-11-27T16:16:12.320 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/138 2022-11-27T16:16:12.320 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/138 2022-11-27T16:16:12.320 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:16:12.321 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:16:12.321 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:16:12.321 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:16:12.321 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/138 2022-11-27T16:16:12.321 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/138 2022-11-27T16:16:12.322 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/138 2022-11-27T16:16:12.322 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/138 2022-11-27T16:16:12.322 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:16:12.322 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/138 2022-11-27T16:16:12.323 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/138 2022-11-27T16:16:12.323 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/138 2022-11-27T16:16:12.323 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/138 2022-11-27T16:16:12.323 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/138 2022-11-27T16:16:12.324 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/138 2022-11-27T16:16:12.324 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/138 2022-11-27T16:16:12.324 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/138 2022-11-27T16:16:12.324 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/138 2022-11-27T16:16:12.325 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/138 2022-11-27T16:16:12.325 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/138 2022-11-27T16:16:12.325 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/138 2022-11-27T16:16:12.325 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/138 2022-11-27T16:16:12.326 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/138 2022-11-27T16:16:12.326 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/138 2022-11-27T16:16:12.326 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/138 2022-11-27T16:16:12.327 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/138 2022-11-27T16:16:12.327 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/138 2022-11-27T16:16:12.327 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/138 2022-11-27T16:16:12.328 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/138 2022-11-27T16:16:12.328 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/138 2022-11-27T16:16:12.328 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/138 2022-11-27T16:16:12.329 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/138 2022-11-27T16:16:12.329 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-11-27T16:16:12.329 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-11-27T16:16:12.330 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-11-27T16:16:12.330 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-11-27T16:16:12.330 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-11-27T16:16:12.330 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-11-27T16:16:12.331 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-11-27T16:16:12.331 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-11-27T16:16:12.332 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-11-27T16:16:12.332 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-11-27T16:16:12.332 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-11-27T16:16:12.333 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-11-27T16:16:12.333 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-11-27T16:16:12.333 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-11-27T16:16:12.334 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-11-27T16:16:12.334 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-11-27T16:16:12.334 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-11-27T16:16:12.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-11-27T16:16:12.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-11-27T16:16:12.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-11-27T16:16:12.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-11-27T16:16:12.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-11-27T16:16:12.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-11-27T16:16:12.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-11-27T16:16:12.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-11-27T16:16:12.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-11-27T16:16:12.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-11-27T16:16:12.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-11-27T16:16:12.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-11-27T16:16:12.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-11-27T16:16:12.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:16:12.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-11-27T16:16:12.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-11-27T16:16:12.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 72/138 2022-11-27T16:16:12.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 73/138 2022-11-27T16:16:12.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 74/138 2022-11-27T16:16:12.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/138 2022-11-27T16:16:12.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 76/138 2022-11-27T16:16:12.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 77/138 2022-11-27T16:16:12.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 78/138 2022-11-27T16:16:12.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 79/138 2022-11-27T16:16:12.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 80/138 2022-11-27T16:16:12.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 81/138 2022-11-27T16:16:12.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/138 2022-11-27T16:16:12.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 83/138 2022-11-27T16:16:12.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 84/138 2022-11-27T16:16:12.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/138 2022-11-27T16:16:12.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 86/138 2022-11-27T16:16:12.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/138 2022-11-27T16:16:12.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 88/138 2022-11-27T16:16:12.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 89/138 2022-11-27T16:16:12.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 90/138 2022-11-27T16:16:12.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 91/138 2022-11-27T16:16:12.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 92/138 2022-11-27T16:16:12.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 93/138 2022-11-27T16:16:12.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 94/138 2022-11-27T16:16:12.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 95/138 2022-11-27T16:16:12.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/138 2022-11-27T16:16:12.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/138 2022-11-27T16:16:12.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 98/138 2022-11-27T16:16:12.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.0.4-10.el8.x86_64 99/138 2022-11-27T16:16:12.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 100/138 2022-11-27T16:16:12.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 101/138 2022-11-27T16:16:12.348 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 102/138 2022-11-27T16:16:12.348 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 103/138 2022-11-27T16:16:12.348 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/138 2022-11-27T16:16:12.348 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 105/138 2022-11-27T16:16:12.348 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 106/138 2022-11-27T16:16:12.349 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 107/138 2022-11-27T16:16:12.349 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/138 2022-11-27T16:16:12.349 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 109/138 2022-11-27T16:16:12.349 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 110/138 2022-11-27T16:16:12.349 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 111/138 2022-11-27T16:16:12.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/138 2022-11-27T16:16:12.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/138 2022-11-27T16:16:12.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 114/138 2022-11-27T16:16:12.350 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.1-9.el8.x86_64 115/138 2022-11-27T16:16:12.351 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 116/138 2022-11-27T16:16:12.351 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.6-3.el8.x86_64 117/138 2022-11-27T16:16:12.351 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 118/138 2022-11-27T16:16:12.351 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.1-7.el8.noarch 119/138 2022-11-27T16:16:12.351 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 120/138 2022-11-27T16:16:12.352 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 121/138 2022-11-27T16:16:12.352 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 122/138 2022-11-27T16:16:12.352 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/138 2022-11-27T16:16:12.352 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 124/138 2022-11-27T16:16:12.353 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 125/138 2022-11-27T16:16:12.353 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 126/138 2022-11-27T16:16:12.353 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 127/138 2022-11-27T16:16:12.353 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 128/138 2022-11-27T16:16:12.353 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 129/138 2022-11-27T16:16:12.354 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-11-27T16:16:12.354 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 131/138 2022-11-27T16:16:12.354 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-11-27T16:16:12.354 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-11-27T16:16:12.354 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:16:12.355 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-11-27T16:16:12.355 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-11-27T16:16:12.355 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-11-27T16:16:12.521 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 126/138 2022-11-27T16:16:12.771 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 127/138 2022-11-27T16:16:13.140 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-11-27T16:16:13.449 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-11-27T16:16:13.639 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T16:16:13.648 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T16:16:13.799 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/138 2022-11-27T16:16:14.013 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T16:16:14.089 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T16:16:14.222 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:16:14.223 INFO:teuthology.orchestra.run.smithi191.stdout:Installed products updated. 2022-11-27T16:16:14.297 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T16:16:14.311 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T16:16:14.460 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:14.461 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2022-11-27T16:16:14.461 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-11-27T16:16:14.461 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.461 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.462 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:16:14.462 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:16:14.462 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2022-11-27T16:16:14.462 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-11-27T16:16:14.463 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T16:16:14.463 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T16:16:14.464 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.464 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.464 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.464 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.464 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:14.465 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.465 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.465 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.465 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:14.466 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:14.466 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:14.466 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:14.466 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:14.467 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.467 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.467 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:14.467 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.467 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.468 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.468 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:14.468 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:14.468 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.469 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:14.469 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-11-27T16:16:14.469 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T16:16:14.469 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.7-15.el8.x86_64 2022-11-27T16:16:14.469 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T16:16:14.470 INFO:teuthology.orchestra.run.smithi191.stdout: jq-1.6-3.el8.x86_64 2022-11-27T16:16:14.470 INFO:teuthology.orchestra.run.smithi191.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T16:16:14.470 INFO:teuthology.orchestra.run.smithi191.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-11-27T16:16:14.470 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T16:16:14.471 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.471 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.471 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.471 INFO:teuthology.orchestra.run.smithi191.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-11-27T16:16:14.471 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:16:14.472 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T16:16:14.472 INFO:teuthology.orchestra.run.smithi191.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T16:16:14.472 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:16:14.472 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T16:16:14.473 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.473 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.473 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T16:16:14.473 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.474 INFO:teuthology.orchestra.run.smithi191.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-11-27T16:16:14.474 INFO:teuthology.orchestra.run.smithi191.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T16:16:14.474 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T16:16:14.474 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T16:16:14.475 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-11-27T16:16:14.475 INFO:teuthology.orchestra.run.smithi191.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-11-27T16:16:14.475 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T16:16:14.475 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T16:16:14.476 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.15-3.el8.x86_64 2022-11-27T16:16:14.476 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-11-27T16:16:14.476 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:16:14.476 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-11-27T16:16:14.476 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-11-27T16:16:14.477 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-11-27T16:16:14.477 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:16:14.477 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:16:14.477 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:16:14.478 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T16:16:14.478 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T16:16:14.478 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T16:16:14.478 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T16:16:14.479 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T16:16:14.479 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.479 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.479 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.479 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T16:16:14.480 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T16:16:14.480 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T16:16:14.480 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T16:16:14.480 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T16:16:14.481 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T16:16:14.481 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T16:16:14.481 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T16:16:14.481 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T16:16:14.481 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T16:16:14.482 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T16:16:14.482 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T16:16:14.482 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T16:16:14.482 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T16:16:14.483 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T16:16:14.483 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T16:16:14.483 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T16:16:14.483 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T16:16:14.483 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T16:16:14.484 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T16:16:14.484 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T16:16:14.484 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T16:16:14.484 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T16:16:14.485 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T16:16:14.485 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T16:16:14.485 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T16:16:14.485 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T16:16:14.485 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T16:16:14.486 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T16:16:14.486 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.486 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.486 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T16:16:14.486 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T16:16:14.487 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.487 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T16:16:14.487 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T16:16:14.487 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-11-27T16:16:14.488 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T16:16:14.488 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T16:16:14.488 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T16:16:14.488 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T16:16:14.488 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T16:16:14.489 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T16:16:14.489 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T16:16:14.489 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T16:16:14.489 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T16:16:14.489 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T16:16:14.490 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.490 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.490 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:14.490 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T16:16:14.491 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T16:16:14.491 INFO:teuthology.orchestra.run.smithi191.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T16:16:14.491 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T16:16:14.491 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T16:16:14.492 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-11-27T16:16:14.492 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:14.493 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2022-11-27T16:16:14.493 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:16:14.493 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:16:14.639 DEBUG:teuthology.parallel:result is None 2022-11-27T16:16:14.667 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:16:14.667 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:16:14.890 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:16:14.890 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:16:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:16:15.345 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:16:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:16:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/138 2022-11-27T16:16:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/138 2022-11-27T16:16:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/138 2022-11-27T16:16:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/138 2022-11-27T16:16:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/138 2022-11-27T16:16:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/138 2022-11-27T16:16:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:16:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:16:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:16:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:16:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/138 2022-11-27T16:16:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/138 2022-11-27T16:16:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/138 2022-11-27T16:16:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/138 2022-11-27T16:16:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:16:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/138 2022-11-27T16:16:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/138 2022-11-27T16:16:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/138 2022-11-27T16:16:15.881 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/138 2022-11-27T16:16:15.881 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/138 2022-11-27T16:16:15.881 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/138 2022-11-27T16:16:15.881 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/138 2022-11-27T16:16:15.881 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/138 2022-11-27T16:16:15.882 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/138 2022-11-27T16:16:15.882 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/138 2022-11-27T16:16:15.882 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/138 2022-11-27T16:16:15.882 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/138 2022-11-27T16:16:15.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/138 2022-11-27T16:16:15.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/138 2022-11-27T16:16:15.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/138 2022-11-27T16:16:15.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/138 2022-11-27T16:16:15.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/138 2022-11-27T16:16:15.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/138 2022-11-27T16:16:15.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/138 2022-11-27T16:16:15.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/138 2022-11-27T16:16:15.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/138 2022-11-27T16:16:15.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/138 2022-11-27T16:16:15.899 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/138 2022-11-27T16:16:15.899 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-11-27T16:16:15.899 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-11-27T16:16:15.899 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-11-27T16:16:15.900 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-11-27T16:16:15.900 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-11-27T16:16:15.900 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-11-27T16:16:15.900 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-11-27T16:16:15.901 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-11-27T16:16:15.901 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-11-27T16:16:15.901 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-11-27T16:16:15.902 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-11-27T16:16:15.902 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-11-27T16:16:15.902 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-11-27T16:16:15.903 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-11-27T16:16:15.903 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-11-27T16:16:15.903 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-11-27T16:16:15.904 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-11-27T16:16:15.904 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-11-27T16:16:15.904 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-11-27T16:16:15.904 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-11-27T16:16:15.904 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-11-27T16:16:15.905 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-11-27T16:16:15.905 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-11-27T16:16:15.905 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-11-27T16:16:15.905 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-11-27T16:16:15.906 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-11-27T16:16:15.906 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-11-27T16:16:15.906 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-11-27T16:16:15.907 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-11-27T16:16:15.907 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-11-27T16:16:15.908 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:16:15.908 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-11-27T16:16:15.908 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-11-27T16:16:15.909 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 72/138 2022-11-27T16:16:15.909 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.0.4-10.el8.x86_64 73/138 2022-11-27T16:16:15.910 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 74/138 2022-11-27T16:16:15.910 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 75/138 2022-11-27T16:16:15.911 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 76/138 2022-11-27T16:16:15.911 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/138 2022-11-27T16:16:15.911 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 78/138 2022-11-27T16:16:15.912 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 79/138 2022-11-27T16:16:15.912 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 80/138 2022-11-27T16:16:15.912 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 81/138 2022-11-27T16:16:15.912 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 82/138 2022-11-27T16:16:15.913 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 83/138 2022-11-27T16:16:15.913 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 84/138 2022-11-27T16:16:15.913 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 85/138 2022-11-27T16:16:15.913 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/138 2022-11-27T16:16:15.913 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 87/138 2022-11-27T16:16:15.914 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/138 2022-11-27T16:16:15.914 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.1-9.el8.x86_64 89/138 2022-11-27T16:16:15.914 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 90/138 2022-11-27T16:16:15.914 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : jq-1.6-3.el8.x86_64 91/138 2022-11-27T16:16:15.915 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 92/138 2022-11-27T16:16:15.915 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.1-7.el8.noarch 93/138 2022-11-27T16:16:15.915 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 94/138 2022-11-27T16:16:15.915 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 95/138 2022-11-27T16:16:15.916 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 96/138 2022-11-27T16:16:15.916 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpq-13.5-1.el8.x86_64 97/138 2022-11-27T16:16:15.916 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 98/138 2022-11-27T16:16:15.916 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 99/138 2022-11-27T16:16:15.917 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 100/138 2022-11-27T16:16:15.917 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 101/138 2022-11-27T16:16:15.917 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/138 2022-11-27T16:16:15.917 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 103/138 2022-11-27T16:16:15.917 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 104/138 2022-11-27T16:16:15.918 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 105/138 2022-11-27T16:16:15.918 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/138 2022-11-27T16:16:15.918 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 107/138 2022-11-27T16:16:15.918 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 108/138 2022-11-27T16:16:15.919 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 109/138 2022-11-27T16:16:15.919 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 110/138 2022-11-27T16:16:15.919 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 111/138 2022-11-27T16:16:15.919 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 112/138 2022-11-27T16:16:15.920 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 113/138 2022-11-27T16:16:15.920 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 114/138 2022-11-27T16:16:15.920 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 115/138 2022-11-27T16:16:15.920 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/138 2022-11-27T16:16:15.921 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 117/138 2022-11-27T16:16:15.921 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/138 2022-11-27T16:16:15.921 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 119/138 2022-11-27T16:16:15.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 120/138 2022-11-27T16:16:15.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 121/138 2022-11-27T16:16:15.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 122/138 2022-11-27T16:16:15.922 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 123/138 2022-11-27T16:16:15.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 124/138 2022-11-27T16:16:15.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 125/138 2022-11-27T16:16:15.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 126/138 2022-11-27T16:16:15.923 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 127/138 2022-11-27T16:16:15.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/138 2022-11-27T16:16:15.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 129/138 2022-11-27T16:16:15.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-11-27T16:16:15.924 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 131/138 2022-11-27T16:16:15.925 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-11-27T16:16:15.925 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-11-27T16:16:15.925 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:16:15.925 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-11-27T16:16:15.926 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-11-27T16:16:15.926 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-11-27T16:16:17.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:16:17.872 INFO:teuthology.orchestra.run.smithi134.stdout:Installed products updated. 2022-11-27T16:16:18.149 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:18.149 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2022-11-27T16:16:18.149 INFO:teuthology.orchestra.run.smithi134.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-11-27T16:16:18.150 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.150 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.150 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:16:18.151 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:16:18.151 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2022-11-27T16:16:18.151 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-11-27T16:16:18.151 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T16:16:18.152 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T16:16:18.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.152 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.153 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.153 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:18.153 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.153 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.154 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.154 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:18.154 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:18.154 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:18.154 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:18.155 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:18.156 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.156 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.156 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:18.156 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.157 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.157 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.157 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:18.157 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:18.158 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.158 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:16:18.158 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-11-27T16:16:18.158 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T16:16:18.159 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.7-15.el8.x86_64 2022-11-27T16:16:18.159 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T16:16:18.160 INFO:teuthology.orchestra.run.smithi134.stdout: jq-1.6-3.el8.x86_64 2022-11-27T16:16:18.160 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T16:16:18.161 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-11-27T16:16:18.161 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T16:16:18.161 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.162 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.162 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.162 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-11-27T16:16:18.163 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:16:18.163 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T16:16:18.163 INFO:teuthology.orchestra.run.smithi134.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T16:16:18.164 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:16:18.164 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T16:16:18.164 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.165 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.165 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T16:16:18.165 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.165 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-11-27T16:16:18.166 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T16:16:18.166 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T16:16:18.166 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T16:16:18.166 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-11-27T16:16:18.167 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-11-27T16:16:18.167 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T16:16:18.167 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T16:16:18.167 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.15-3.el8.x86_64 2022-11-27T16:16:18.168 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-11-27T16:16:18.168 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:16:18.168 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-11-27T16:16:18.168 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-11-27T16:16:18.169 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-11-27T16:16:18.169 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:16:18.169 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:16:18.170 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:16:18.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T16:16:18.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T16:16:18.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T16:16:18.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T16:16:18.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T16:16:18.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.173 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T16:16:18.173 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T16:16:18.174 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T16:16:18.174 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T16:16:18.174 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T16:16:18.174 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T16:16:18.174 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T16:16:18.175 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T16:16:18.175 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T16:16:18.175 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T16:16:18.175 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T16:16:18.176 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T16:16:18.176 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T16:16:18.176 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T16:16:18.177 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T16:16:18.177 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T16:16:18.177 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T16:16:18.177 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T16:16:18.178 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T16:16:18.178 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T16:16:18.178 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T16:16:18.179 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T16:16:18.179 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T16:16:18.179 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T16:16:18.180 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T16:16:18.180 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T16:16:18.180 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T16:16:18.181 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T16:16:18.181 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T16:16:18.181 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.181 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.182 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T16:16:18.182 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T16:16:18.182 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.182 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T16:16:18.182 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T16:16:18.183 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-11-27T16:16:18.183 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T16:16:18.183 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T16:16:18.184 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T16:16:18.184 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T16:16:18.184 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T16:16:18.185 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T16:16:18.185 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T16:16:18.186 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T16:16:18.186 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T16:16:18.186 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T16:16:18.187 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.187 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.187 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:16:18.188 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T16:16:18.188 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T16:16:18.188 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T16:16:18.189 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T16:16:18.189 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T16:16:18.190 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-11-27T16:16:18.190 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:18.190 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2022-11-27T16:16:18.348 DEBUG:teuthology.parallel:result is None 2022-11-27T16:16:18.350 DEBUG:teuthology.orchestra.run.smithi134:> sudo lsb_release -is 2022-11-27T16:16:18.406 INFO:teuthology.orchestra.run.smithi134.stdout:RedHatEnterprise 2022-11-27T16:16:18.407 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:16:18.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:16:18.590 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T16:16:18.612 INFO:teuthology.orchestra.run.smithi134.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T16:16:18.613 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T16:16:18.613 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T16:16:18.614 DEBUG:teuthology.orchestra.run.smithi191:> sudo lsb_release -is 2022-11-27T16:16:18.680 INFO:teuthology.orchestra.run.smithi191.stdout:RedHatEnterprise 2022-11-27T16:16:18.682 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:16:18.683 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:16:18.828 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T16:16:18.854 INFO:teuthology.orchestra.run.smithi191.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T16:16:18.854 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T16:16:18.855 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T16:16:18.855 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-11-27T16:16:18.856 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:18.856 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T16:16:18.883 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:18.884 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T16:16:18.925 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-11-27T16:16:18.925 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:18.926 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T16:16:18.952 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T16:16:19.019 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:19.019 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T16:16:19.048 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T16:16:19.116 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-11-27T16:16:19.117 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:19.117 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T16:16:19.144 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T16:16:19.210 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:19.210 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T16:16:19.244 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T16:16:19.311 INFO:teuthology.run_tasks:Running task ceph... 2022-11-27T16:16:19.378 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-11-27T16:16:19.379 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /var/log/ceph 2022-11-27T16:16:19.382 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /var/log/ceph 2022-11-27T16:16:19.409 INFO:tasks.ceph:Disabling ceph logrotate... 2022-11-27T16:16:19.409 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T16:16:19.449 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T16:16:19.475 INFO:tasks.ceph:Creating extra log directories... 2022-11-27T16:16:19.476 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T16:16:19.516 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T16:16:19.585 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-11-27T16:16:19.585 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-11-27T16:16:19.586 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-11-27T16:16:19.587 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:19.587 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T16:16:19.605 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T16:16:19.729 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T16:16:19.756 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:19.757 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T16:16:19.774 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T16:16:19.887 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T16:16:19.920 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-11-27T16:16:19.921 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-11-27T16:16:19.921 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088644', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '7088644', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi134.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZKu7FffAcWUek5T7f+sB4QCRONCfIL/vSKhrwExRvmnF9N+QHTo8HIQr+bYBSmWQIkRiKZZEOuLgnd+S4W9kehQfVaeYjyA2p9noW72reaPThszR2sxKJLh9IFMcIlbDN1Xleo5mL25TdKbB6xDc42DO+DnOtvpotkSwE2etew+xzTt9kMQRNofF2jVkuHzYpCGA4liOZzQYsqK0pc/QUnqcxLP/YS6KDl57n53bl1ltO9shTTIZ4xhq3tsQtqT5JBd3sLyll5+PGl2u2UW1IgHKy+dU2OfXEzrBMosy6Bi1Ud72up8wkrYiLaRgst3GYE7tqe8cLcs8xDkF2749gvjPwREY6ptGjESOM0LHLTXn10DqVxjYmOY/zFqX0t+xCjvHszFh/yyzssdTJk/9rcCMMw0AR3BM0pIKYRAV96i8wMK/I7rvWSuk0kDzW1uqSQWgGjklOIbWwepMvBZb0sDOzoqgYO7w/4VAfOZm6z9NIxACxpKSDDb80Rn8ZAAM=', 'smithi191.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLA9R28odf0ze2bQSa9JOMXiJ+7o1/SEEkiimPC5DzhKKgL0I9ZUMOMUuuI64CwctuxdvNZpN/4fGxDb+sMQAiA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_data_scan']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-23_17:24:26', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23126'} 2022-11-27T16:16:19.922 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T16:16:19.925 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T16:16:19.971 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T16:16:19.982 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T16:16:20.039 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:20.039 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T16:16:20.064 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T16:16:20.065 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2022-11-27T16:16:20.123 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T16:16:20.123 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:16:20.123 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 359235 Links: 1 2022-11-27T16:16:20.124 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:16:20.124 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:16:20.124 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2022-11-27 16:16:15.555732516 +0000 2022-11-27T16:16:20.124 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2022-11-27 16:13:27.288746476 +0000 2022-11-27T16:16:20.125 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2022-11-27 16:13:27.288746476 +0000 2022-11-27T16:16:20.125 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2022-11-27T16:16:20.125 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T16:16:20.189 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2022-11-27T16:16:20.189 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2022-11-27T16:16:20.190 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000221438 s, 2.3 MB/s 2022-11-27T16:16:20.191 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T16:16:20.249 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2022-11-27T16:16:20.307 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T16:16:20.307 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:16:20.307 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 363746 Links: 1 2022-11-27T16:16:20.307 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:16:20.308 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:16:20.308 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2022-11-27 16:16:15.555732516 +0000 2022-11-27T16:16:20.308 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2022-11-27 16:13:27.839733274 +0000 2022-11-27T16:16:20.308 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2022-11-27 16:13:27.839733274 +0000 2022-11-27T16:16:20.309 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2022-11-27T16:16:20.309 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T16:16:20.372 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2022-11-27T16:16:20.372 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2022-11-27T16:16:20.372 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000187447 s, 2.7 MB/s 2022-11-27T16:16:20.373 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T16:16:20.431 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2022-11-27T16:16:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T16:16:20.489 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:16:20.490 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 351150 Links: 1 2022-11-27T16:16:20.490 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:16:20.491 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:16:20.491 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2022-11-27 16:16:15.555732516 +0000 2022-11-27T16:16:20.491 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2022-11-27 16:13:28.388720119 +0000 2022-11-27T16:16:20.492 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2022-11-27 16:13:28.388720119 +0000 2022-11-27T16:16:20.492 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2022-11-27T16:16:20.493 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T16:16:20.554 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2022-11-27T16:16:20.555 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2022-11-27T16:16:20.555 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000214135 s, 2.4 MB/s 2022-11-27T16:16:20.557 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T16:16:20.615 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2022-11-27T16:16:20.673 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T16:16:20.673 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:16:20.674 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 366821 Links: 1 2022-11-27T16:16:20.674 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:16:20.674 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:16:20.674 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2022-11-27 16:16:15.555732516 +0000 2022-11-27T16:16:20.674 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2022-11-27 16:13:29.095703180 +0000 2022-11-27T16:16:20.675 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2022-11-27 16:13:29.095703180 +0000 2022-11-27T16:16:20.675 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2022-11-27T16:16:20.675 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T16:16:20.738 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2022-11-27T16:16:20.739 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2022-11-27T16:16:20.739 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000220382 s, 2.3 MB/s 2022-11-27T16:16:20.740 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T16:16:20.798 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:16:20.799 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:20.799 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T16:16:20.818 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T16:16:20.818 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2022-11-27T16:16:20.876 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T16:16:20.876 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:16:20.877 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 362818 Links: 1 2022-11-27T16:16:20.877 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:16:20.877 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:16:20.877 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2022-11-27 16:16:11.978980840 +0000 2022-11-27T16:16:20.878 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2022-11-27 16:13:27.310835677 +0000 2022-11-27T16:16:20.878 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2022-11-27 16:13:27.310835677 +0000 2022-11-27T16:16:20.878 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2022-11-27T16:16:20.878 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T16:16:20.943 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2022-11-27T16:16:20.944 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2022-11-27T16:16:20.944 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000231698 s, 2.2 MB/s 2022-11-27T16:16:20.946 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T16:16:21.006 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2022-11-27T16:16:21.065 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T16:16:21.065 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:16:21.066 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 365821 Links: 1 2022-11-27T16:16:21.066 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:16:21.067 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:16:21.067 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2022-11-27 16:16:11.978980840 +0000 2022-11-27T16:16:21.067 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2022-11-27 16:13:27.870826059 +0000 2022-11-27T16:16:21.068 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2022-11-27 16:13:27.870826059 +0000 2022-11-27T16:16:21.068 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2022-11-27T16:16:21.069 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T16:16:21.132 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2022-11-27T16:16:21.132 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2022-11-27T16:16:21.132 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.00022241 s, 2.3 MB/s 2022-11-27T16:16:21.133 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T16:16:21.192 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2022-11-27T16:16:21.251 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T16:16:21.251 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:16:21.251 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 365882 Links: 1 2022-11-27T16:16:21.252 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:16:21.252 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:16:21.252 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2022-11-27 16:16:11.979980823 +0000 2022-11-27T16:16:21.252 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2022-11-27 16:13:28.425816527 +0000 2022-11-27T16:16:21.252 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2022-11-27 16:13:28.425816527 +0000 2022-11-27T16:16:21.253 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2022-11-27T16:16:21.253 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T16:16:21.318 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2022-11-27T16:16:21.320 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2022-11-27T16:16:21.320 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000216555 s, 2.4 MB/s 2022-11-27T16:16:21.322 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T16:16:21.381 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2022-11-27T16:16:21.439 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T16:16:21.439 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:16:21.440 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 362917 Links: 1 2022-11-27T16:16:21.440 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:16:21.440 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:16:21.440 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2022-11-27 16:16:11.979980823 +0000 2022-11-27T16:16:21.441 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2022-11-27 16:13:28.960807338 +0000 2022-11-27T16:16:21.441 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2022-11-27 16:13:28.960807338 +0000 2022-11-27T16:16:21.441 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2022-11-27T16:16:21.442 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T16:16:21.505 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2022-11-27T16:16:21.505 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2022-11-27T16:16:21.505 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.00024043 s, 2.1 MB/s 2022-11-27T16:16:21.507 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T16:16:21.564 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:16:21.565 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T16:16:21.565 INFO:tasks.ceph:Generating config... 2022-11-27T16:16:21.567 INFO:tasks.ceph:[client] client mount timeout = 600 2022-11-27T16:16:21.567 INFO:tasks.ceph:[client] debug client = 20 2022-11-27T16:16:21.568 INFO:tasks.ceph:[client] debug ms = 1 2022-11-27T16:16:21.568 INFO:tasks.ceph:[client] pid file = 2022-11-27T16:16:21.568 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-11-27T16:16:21.569 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-11-27T16:16:21.569 INFO:tasks.ceph:[global] ms die on skipped message = False 2022-11-27T16:16:21.569 INFO:tasks.ceph:[mds] debug mds = 20 2022-11-27T16:16:21.570 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-11-27T16:16:21.570 INFO:tasks.ceph:[mds] debug ms = 1 2022-11-27T16:16:21.571 INFO:tasks.ceph:[mds] mds debug frag = True 2022-11-27T16:16:21.571 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-11-27T16:16:21.571 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-11-27T16:16:21.572 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-11-27T16:16:21.572 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-11-27T16:16:21.573 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-11-27T16:16:21.574 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-11-27T16:16:21.574 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-11-27T16:16:21.575 INFO:tasks.ceph:[mgr] debug ms = 1 2022-11-27T16:16:21.576 INFO:tasks.ceph:[mon] debug mon = 20 2022-11-27T16:16:21.576 INFO:tasks.ceph:[mon] debug ms = 1 2022-11-27T16:16:21.577 INFO:tasks.ceph:[mon] debug paxos = 20 2022-11-27T16:16:21.577 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-11-27T16:16:21.578 INFO:tasks.ceph:[osd] bdev async discard = True 2022-11-27T16:16:21.578 INFO:tasks.ceph:[osd] bdev enable discard = True 2022-11-27T16:16:21.579 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2022-11-27T16:16:21.579 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-11-27T16:16:21.579 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-11-27T16:16:21.580 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2022-11-27T16:16:21.580 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2022-11-27T16:16:21.580 INFO:tasks.ceph:[osd] debug ms = 1 2022-11-27T16:16:21.581 INFO:tasks.ceph:[osd] debug osd = 20 2022-11-27T16:16:21.581 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2022-11-27T16:16:21.581 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-11-27T16:16:21.582 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-11-27T16:16:21.582 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-11-27T16:16:21.582 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-11-27T16:16:21.583 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-11-27T16:16:21.583 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-11-27T16:16:21.583 INFO:tasks.ceph:Setting up mon.a... 2022-11-27T16:16:21.584 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-11-27T16:16:21.642 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.keyring 2022-11-27T16:16:21.645 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-11-27T16:16:21.694 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:16:21.722 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.134'), ('mon.b', '172.21.15.191'), ('mon.c', '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]')] 2022-11-27T16:16:21.723 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.134,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-11-27T16:16:21.725 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:21.725 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T16:16:21.781 DEBUG:teuthology.orchestra.run.smithi134:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.134 --add b 172.21.15.191 --addv c '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:21.871 INFO:teuthology.orchestra.run.smithi134.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:21.872 INFO:teuthology.orchestra.run.smithi134.stdout:monmaptool: generated fsid 34bf4849-eb1e-4259-a7f0-0be8a82c33c9 2022-11-27T16:16:21.872 INFO:teuthology.orchestra.run.smithi134.stdout:setting min_mon_release = pacific 2022-11-27T16:16:21.873 INFO:teuthology.orchestra.run.smithi134.stdout:epoch 0 2022-11-27T16:16:21.873 INFO:teuthology.orchestra.run.smithi134.stdout:fsid 34bf4849-eb1e-4259-a7f0-0be8a82c33c9 2022-11-27T16:16:21.873 INFO:teuthology.orchestra.run.smithi134.stdout:last_changed 2022-11-27T16:16:21.871106+0000 2022-11-27T16:16:21.874 INFO:teuthology.orchestra.run.smithi134.stdout:created 2022-11-27T16:16:21.871106+0000 2022-11-27T16:16:21.874 INFO:teuthology.orchestra.run.smithi134.stdout:min_mon_release 16 (pacific) 2022-11-27T16:16:21.874 INFO:teuthology.orchestra.run.smithi134.stdout:election_strategy: 1 2022-11-27T16:16:21.874 INFO:teuthology.orchestra.run.smithi134.stdout:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2022-11-27T16:16:21.875 INFO:teuthology.orchestra.run.smithi134.stdout:1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2022-11-27T16:16:21.875 INFO:teuthology.orchestra.run.smithi134.stdout:2: [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] mon.c 2022-11-27T16:16:21.875 INFO:teuthology.orchestra.run.smithi134.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-11-27T16:16:21.877 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T16:16:21.894 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 34bf4849-eb1e-4259-a7f0-0be8a82c33c9... 2022-11-27T16:16:21.895 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T16:16:21.937 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T16:16:21.979 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2022-11-27T16:16:21.979 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2022-11-27T16:16:21.979 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T16:16:21.980 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2022-11-27T16:16:21.980 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.980 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2022-11-27T16:16:21.980 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.981 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2022-11-27T16:16:21.981 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.981 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2022-11-27T16:16:21.981 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2022-11-27T16:16:21.982 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.982 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2022-11-27T16:16:21.982 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2022-11-27T16:16:21.982 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.983 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T16:16:21.983 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2022-11-27T16:16:21.983 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.983 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T16:16:21.984 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T16:16:21.984 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.984 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2022-11-27T16:16:21.985 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.985 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2022-11-27T16:16:21.985 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2022-11-27T16:16:21.985 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2022-11-27T16:16:21.986 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2022-11-27T16:16:21.986 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2022-11-27T16:16:21.987 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2022-11-27T16:16:21.987 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2022-11-27T16:16:21.988 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T16:16:21.988 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T16:16:21.988 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2022-11-27T16:16:21.988 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.989 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T16:16:21.989 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.989 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2022-11-27T16:16:21.989 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.989 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2022-11-27T16:16:21.990 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.990 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2022-11-27T16:16:21.990 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2022-11-27T16:16:21.991 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2022-11-27T16:16:21.991 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.134,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2022-11-27T16:16:21.991 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on skipped message = False 2022-11-27T16:16:21.991 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = 34bf4849-eb1e-4259-a7f0-0be8a82c33c9 2022-11-27T16:16:21.991 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.992 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2022-11-27T16:16:21.992 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2022-11-27T16:16:21.992 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.992 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2022-11-27T16:16:21.993 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2022-11-27T16:16:21.993 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.993 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2022-11-27T16:16:21.993 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2022-11-27T16:16:21.993 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.994 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2022-11-27T16:16:21.994 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2022-11-27T16:16:21.994 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2022-11-27T16:16:21.994 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.995 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2022-11-27T16:16:21.995 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2022-11-27T16:16:21.995 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.995 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2022-11-27T16:16:21.995 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.996 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2022-11-27T16:16:21.996 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.996 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2022-11-27T16:16:21.996 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.997 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2022-11-27T16:16:21.997 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2022-11-27T16:16:21.997 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:21.997 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2022-11-27T16:16:21.997 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2022-11-27T16:16:21.998 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2022-11-27T16:16:21.998 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2022-11-27T16:16:21.998 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2022-11-27T16:16:21.998 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2022-11-27T16:16:21.999 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2022-11-27T16:16:21.999 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2022-11-27T16:16:21.999 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2022-11-27T16:16:21.999 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2022-11-27T16:16:21.999 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2022-11-27T16:16:22.000 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2022-11-27T16:16:22.000 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T16:16:22.000 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2022-11-27T16:16:22.000 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2022-11-27T16:16:22.001 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2022-11-27T16:16:22.001 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2022-11-27T16:16:22.001 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2022-11-27T16:16:22.002 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:22.002 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2022-11-27T16:16:22.003 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2022-11-27T16:16:22.003 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2022-11-27T16:16:22.003 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2022-11-27T16:16:22.004 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2022-11-27T16:16:22.004 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:16:22.005 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:16:22.005 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2022-11-27T16:16:22.006 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:22.006 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2022-11-27T16:16:22.006 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2022-11-27T16:16:22.007 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2022-11-27T16:16:22.007 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2022-11-27T16:16:22.008 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2022-11-27T16:16:22.008 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2022-11-27T16:16:22.008 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2022-11-27T16:16:22.009 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:16:22.009 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2022-11-27T16:16:22.009 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2022-11-27T16:16:22.009 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:16:22.010 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:22.010 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T16:16:22.010 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T16:16:22.010 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T16:16:22.011 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:22.011 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T16:16:22.011 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T16:16:22.011 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T16:16:22.011 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:22.012 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2022-11-27T16:16:22.012 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2022-11-27T16:16:22.012 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:22.012 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2022-11-27T16:16:22.012 INFO:teuthology.orchestra.run.smithi134.stdout: mon op complaint time = 120 2022-11-27T16:16:22.013 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:22.013 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2022-11-27T16:16:22.013 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2022-11-27T16:16:22.013 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2022-11-27T16:16:22.014 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2022-11-27T16:16:22.014 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T16:16:22.014 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T16:16:22.014 INFO:teuthology.orchestra.run.smithi134.stdout: client mount timeout = 600 2022-11-27T16:16:22.014 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2022-11-27T16:16:22.015 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2022-11-27T16:16:22.015 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = "" 2022-11-27T16:16:22.015 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2022-11-27T16:16:22.015 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2022-11-27T16:16:22.016 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2022-11-27T16:16:22.016 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2022-11-27T16:16:22.016 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2022-11-27T16:16:22.016 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2022-11-27T16:16:22.016 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2022-11-27T16:16:22.017 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2022-11-27T16:16:22.017 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2022-11-27T16:16:22.017 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2022-11-27T16:16:22.017 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds = 20 2022-11-27T16:16:22.017 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds balancer = 20 2022-11-27T16:16:22.018 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2022-11-27T16:16:22.018 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug frag = True 2022-11-27T16:16:22.018 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug scatterstat = True 2022-11-27T16:16:22.018 INFO:teuthology.orchestra.run.smithi134.stdout: mds op complaint time = 180 2022-11-27T16:16:22.019 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = True 2022-11-27T16:16:22.019 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2022-11-27T16:16:22.019 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2022-11-27T16:16:22.019 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2022-11-27T16:16:22.021 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2022-11-27T16:16:22.021 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2022-11-27T16:16:22.022 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T16:16:22.022 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2022-11-27T16:16:22.022 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.022 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2022-11-27T16:16:22.023 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.023 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2022-11-27T16:16:22.023 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.023 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2022-11-27T16:16:22.024 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2022-11-27T16:16:22.024 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.024 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2022-11-27T16:16:22.024 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2022-11-27T16:16:22.025 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.025 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T16:16:22.025 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2022-11-27T16:16:22.025 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.025 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T16:16:22.026 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T16:16:22.026 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.026 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2022-11-27T16:16:22.026 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.027 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2022-11-27T16:16:22.027 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2022-11-27T16:16:22.027 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2022-11-27T16:16:22.027 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2022-11-27T16:16:22.028 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2022-11-27T16:16:22.028 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2022-11-27T16:16:22.028 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2022-11-27T16:16:22.028 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T16:16:22.028 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T16:16:22.029 INFO:teuthology.orchestra.run.smithi191.stdout: mon_allow_pool_size_one = true 2022-11-27T16:16:22.029 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.029 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T16:16:22.029 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.030 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2022-11-27T16:16:22.030 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.030 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2022-11-27T16:16:22.030 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.030 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2022-11-27T16:16:22.031 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2022-11-27T16:16:22.031 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2022-11-27T16:16:22.031 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.134,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2022-11-27T16:16:22.031 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on skipped message = False 2022-11-27T16:16:22.032 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = 34bf4849-eb1e-4259-a7f0-0be8a82c33c9 2022-11-27T16:16:22.032 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.032 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2022-11-27T16:16:22.032 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2022-11-27T16:16:22.032 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.033 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2022-11-27T16:16:22.033 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2022-11-27T16:16:22.033 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.033 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2022-11-27T16:16:22.034 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2022-11-27T16:16:22.034 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.034 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2022-11-27T16:16:22.034 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2022-11-27T16:16:22.035 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2022-11-27T16:16:22.035 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.035 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2022-11-27T16:16:22.035 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2022-11-27T16:16:22.036 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.036 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2022-11-27T16:16:22.036 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.036 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2022-11-27T16:16:22.036 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.037 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2022-11-27T16:16:22.037 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.037 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2022-11-27T16:16:22.037 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2022-11-27T16:16:22.038 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.038 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2022-11-27T16:16:22.038 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2022-11-27T16:16:22.038 INFO:teuthology.orchestra.run.smithi191.stdout: bdev async discard = True 2022-11-27T16:16:22.039 INFO:teuthology.orchestra.run.smithi191.stdout: bdev enable discard = True 2022-11-27T16:16:22.039 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore allocator = bitmap 2022-11-27T16:16:22.039 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore block size = 96636764160 2022-11-27T16:16:22.039 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore fsck on mount = True 2022-11-27T16:16:22.040 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluefs = 1/20 2022-11-27T16:16:22.040 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluestore = 1/20 2022-11-27T16:16:22.040 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-11-27T16:16:22.040 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2022-11-27T16:16:22.041 INFO:teuthology.orchestra.run.smithi191.stdout: debug rocksdb = 4/10 2022-11-27T16:16:22.041 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T16:16:22.041 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd full ratio = 0.9 2022-11-27T16:16:22.041 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd nearfull ratio = 0.8 2022-11-27T16:16:22.041 INFO:teuthology.orchestra.run.smithi191.stdout: osd failsafe full ratio = 0.95 2022-11-27T16:16:22.042 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = bluestore 2022-11-27T16:16:22.042 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2022-11-27T16:16:22.042 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.042 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2022-11-27T16:16:22.043 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-11-27T16:16:22.043 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2022-11-27T16:16:22.043 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2022-11-27T16:16:22.043 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2022-11-27T16:16:22.044 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:16:22.044 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:16:22.044 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2022-11-27T16:16:22.044 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.044 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2022-11-27T16:16:22.045 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-11-27T16:16:22.045 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2022-11-27T16:16:22.045 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2022-11-27T16:16:22.045 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2022-11-27T16:16:22.045 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2022-11-27T16:16:22.046 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2022-11-27T16:16:22.046 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:16:22.046 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2022-11-27T16:16:22.046 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2022-11-27T16:16:22.047 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:16:22.047 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.047 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T16:16:22.047 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T16:16:22.047 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T16:16:22.048 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.048 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T16:16:22.048 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T16:16:22.048 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T16:16:22.049 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.049 INFO:teuthology.orchestra.run.smithi191.stdout: # 1m isn't quite enough 2022-11-27T16:16:22.049 INFO:teuthology.orchestra.run.smithi191.stdout: mon_down_mkfs_grace = 2m 2022-11-27T16:16:22.049 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.049 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_filestore_osds = false 2022-11-27T16:16:22.050 INFO:teuthology.orchestra.run.smithi191.stdout: mon op complaint time = 120 2022-11-27T16:16:22.050 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-11-27T16:16:22.051 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2022-11-27T16:16:22.051 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2022-11-27T16:16:22.051 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2022-11-27T16:16:22.051 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2022-11-27T16:16:22.051 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T16:16:22.052 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T16:16:22.052 INFO:teuthology.orchestra.run.smithi191.stdout: client mount timeout = 600 2022-11-27T16:16:22.052 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2022-11-27T16:16:22.052 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-11-27T16:16:22.053 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = "" 2022-11-27T16:16:22.053 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2022-11-27T16:16:22.053 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2022-11-27T16:16:22.053 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2022-11-27T16:16:22.053 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2022-11-27T16:16:22.054 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2022-11-27T16:16:22.054 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2022-11-27T16:16:22.054 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2022-11-27T16:16:22.054 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2022-11-27T16:16:22.055 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2022-11-27T16:16:22.055 INFO:teuthology.orchestra.run.smithi191.stdout:[mds] 2022-11-27T16:16:22.055 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds = 20 2022-11-27T16:16:22.055 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds balancer = 20 2022-11-27T16:16:22.056 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-11-27T16:16:22.056 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug frag = True 2022-11-27T16:16:22.056 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug scatterstat = True 2022-11-27T16:16:22.056 INFO:teuthology.orchestra.run.smithi191.stdout: mds op complaint time = 180 2022-11-27T16:16:22.056 INFO:teuthology.orchestra.run.smithi191.stdout: mds verify scatter = True 2022-11-27T16:16:22.057 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2022-11-27T16:16:22.057 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2022-11-27T16:16:22.057 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2022-11-27T16:16:22.057 INFO:tasks.ceph:Creating admin key on mon.a... 2022-11-27T16:16:22.058 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-11-27T16:16:22.118 INFO:tasks.ceph:Copying monmap to all nodes... 2022-11-27T16:16:22.119 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:22.119 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-11-27T16:16:22.139 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:22.140 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-11-27T16:16:22.196 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi134.front.sepia.ceph.com 2022-11-27T16:16:22.197 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:22.197 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T16:16:22.198 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:16:22.273 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:22.273 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:22.330 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi191.front.sepia.ceph.com 2022-11-27T16:16:22.331 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:22.331 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T16:16:22.331 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:16:22.370 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:22.370 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:22.427 INFO:tasks.ceph:Setting up mon nodes... 2022-11-27T16:16:22.427 INFO:tasks.ceph:Setting up mgr nodes... 2022-11-27T16:16:22.428 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T16:16:22.481 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T16:16:22.484 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T16:16:22.551 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T16:16:22.554 INFO:tasks.ceph:Setting up mds nodes... 2022-11-27T16:16:22.555 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T16:16:22.608 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T16:16:22.611 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-11-27T16:16:22.637 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T16:16:22.730 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T16:16:22.733 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-11-27T16:16:22.760 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T16:16:22.814 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T16:16:22.817 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-11-27T16:16:22.844 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T16:16:22.937 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T16:16:22.940 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2022-11-27T16:16:22.966 INFO:tasks.ceph_client:Setting up client nodes... 2022-11-27T16:16:22.967 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-11-27T16:16:23.011 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-11-27T16:16:23.024 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2022-11-27T16:16:23.107 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.2.keyring 2022-11-27T16:16:23.121 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-11-27T16:16:23.166 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-11-27T16:16:23.179 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2022-11-27T16:16:23.262 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.3.keyring 2022-11-27T16:16:23.275 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-11-27T16:16:23.275 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T16:16:23.276 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-11-27T16:16:23.303 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:16:23.303 INFO:tasks.ceph:role: osd.0 2022-11-27T16:16:23.304 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com 2022-11-27T16:16:23.304 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T16:16:23.377 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:16:23.378 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:16:23.378 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:16:23.378 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:16:23.378 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:16:23.379 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:16:23.379 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:16:23.379 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:16:23.379 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:16:23.379 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:16:23.805 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2022-11-27T16:16:23.807 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2022-11-27T16:16:23.807 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-11-27T16:16:23.982 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-11-27T16:16:24.016 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-11-27T16:16:24.084 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:16:24.084 INFO:tasks.ceph:role: osd.1 2022-11-27T16:16:24.085 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com 2022-11-27T16:16:24.086 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T16:16:24.158 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:16:24.158 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:16:24.159 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:16:24.159 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:16:24.160 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:16:24.160 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:16:24.161 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:16:24.161 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:16:24.161 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:16:24.162 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:16:24.581 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2022-11-27T16:16:24.584 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2022-11-27T16:16:24.584 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-11-27T16:16:24.691 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-11-27T16:16:24.722 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-11-27T16:16:24.789 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:16:24.790 INFO:tasks.ceph:role: osd.2 2022-11-27T16:16:24.790 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com 2022-11-27T16:16:24.790 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T16:16:24.863 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:16:24.864 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:16:24.864 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:16:24.864 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:16:24.865 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:16:24.865 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:16:24.865 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:16:24.866 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:16:24.866 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:16:24.866 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:16:25.304 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2022-11-27T16:16:25.306 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2022-11-27T16:16:25.306 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-11-27T16:16:25.451 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-11-27T16:16:25.486 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-11-27T16:16:25.552 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:16:25.553 INFO:tasks.ceph:role: osd.3 2022-11-27T16:16:25.553 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com 2022-11-27T16:16:25.554 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T16:16:25.627 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:16:25.628 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:16:25.628 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:16:25.628 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:16:25.628 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:16:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:16:25.629 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:16:25.629 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:16:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:16:25.630 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:16:26.073 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2022-11-27T16:16:26.076 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2022-11-27T16:16:26.076 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-11-27T16:16:26.262 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-11-27T16:16:26.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:26.393 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:26.391+0000 7f714b18f5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-11-27T16:16:26.394 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:26.391+0000 7f714b18f5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-11-27T16:16:26.394 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:26.391+0000 7f714b18f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T16:16:26.395 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:26.391+0000 7f714b18f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T16:16:26.395 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:26.391+0000 7f714b18f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T16:16:26.395 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:26.392+0000 7f714b18f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-11-27T16:16:29.251 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-11-27T16:16:29.283 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:29.376+0000 7f47c030e5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-11-27T16:16:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:29.376+0000 7f47c030e5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-11-27T16:16:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:29.376+0000 7f47c030e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T16:16:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:29.376+0000 7f47c030e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T16:16:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:29.376+0000 7f47c030e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T16:16:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:29.377+0000 7f47c030e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-11-27T16:16:32.243 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-11-27T16:16:32.277 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:32.369 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:32.366+0000 7fb6f7e4b5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-11-27T16:16:32.369 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:32.367+0000 7fb6f7e4b5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-11-27T16:16:32.369 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:32.367+0000 7fb6f7e4b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T16:16:32.370 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:32.367+0000 7fb6f7e4b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T16:16:32.370 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:32.367+0000 7fb6f7e4b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T16:16:32.370 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:32.367+0000 7fb6f7e4b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-11-27T16:16:35.231 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-11-27T16:16:35.263 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:35.352+0000 7f4c872865c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-11-27T16:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:35.353+0000 7f4c872865c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-11-27T16:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:35.353+0000 7f4c872865c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T16:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:35.353+0000 7f4c872865c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T16:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:35.353+0000 7f4c872865c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T16:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:35.353+0000 7f4c872865c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-11-27T16:16:38.213 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-11-27T16:16:38.245 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-11-27T16:16:38.279 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:16:38.279 INFO:tasks.ceph:role: osd.4 2022-11-27T16:16:38.279 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com 2022-11-27T16:16:38.280 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T16:16:38.356 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:16:38.357 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:16:38.357 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:16:38.358 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:16:38.358 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:16:38.358 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:16:38.359 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:16:38.359 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:16:38.360 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:16:38.360 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:16:38.785 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2022-11-27T16:16:38.787 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2022-11-27T16:16:38.787 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-11-27T16:16:38.965 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-11-27T16:16:39.005 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-11-27T16:16:39.082 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:16:39.083 INFO:tasks.ceph:role: osd.5 2022-11-27T16:16:39.083 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com 2022-11-27T16:16:39.083 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T16:16:39.158 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:16:39.159 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:16:39.159 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:16:39.159 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:16:39.160 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:16:39.160 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:16:39.160 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:16:39.160 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:16:39.160 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:16:39.161 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:16:39.578 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2022-11-27T16:16:39.580 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2022-11-27T16:16:39.580 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-11-27T16:16:39.691 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-11-27T16:16:39.725 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-11-27T16:16:39.795 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:16:39.796 INFO:tasks.ceph:role: osd.6 2022-11-27T16:16:39.796 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com 2022-11-27T16:16:39.796 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T16:16:39.871 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:16:39.872 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:16:39.872 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:16:39.872 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:16:39.872 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:16:39.873 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:16:39.873 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:16:39.873 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:16:39.873 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:16:39.874 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:16:40.314 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2022-11-27T16:16:40.316 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2022-11-27T16:16:40.317 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-11-27T16:16:40.468 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-11-27T16:16:40.504 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-11-27T16:16:40.573 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:16:40.573 INFO:tasks.ceph:role: osd.7 2022-11-27T16:16:40.573 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com 2022-11-27T16:16:40.574 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T16:16:40.649 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:16:40.650 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:16:40.650 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:16:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:16:40.651 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:16:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:16:40.651 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:16:40.652 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:16:40.652 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:16:40.652 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:16:41.091 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2022-11-27T16:16:41.099 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2022-11-27T16:16:41.100 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-11-27T16:16:41.288 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-11-27T16:16:41.325 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:41.416 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:41.414+0000 7f7af04ff5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-11-27T16:16:41.417 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:41.414+0000 7f7af04ff5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-11-27T16:16:41.417 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:41.414+0000 7f7af04ff5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T16:16:41.418 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:41.414+0000 7f7af04ff5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T16:16:41.418 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:41.414+0000 7f7af04ff5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T16:16:41.418 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:41.415+0000 7f7af04ff5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-11-27T16:16:44.278 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-11-27T16:16:44.307 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:44.398 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:44.395+0000 7fcc4d1ec5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-11-27T16:16:44.398 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:44.395+0000 7fcc4d1ec5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-11-27T16:16:44.399 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:44.395+0000 7fcc4d1ec5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T16:16:44.399 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:44.395+0000 7fcc4d1ec5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T16:16:44.399 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:44.395+0000 7fcc4d1ec5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T16:16:44.400 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:44.396+0000 7fcc4d1ec5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-11-27T16:16:47.268 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-11-27T16:16:47.299 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:47.388 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:47.386+0000 7f99fe1665c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-11-27T16:16:47.389 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:47.386+0000 7f99fe1665c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-11-27T16:16:47.389 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:47.386+0000 7f99fe1665c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T16:16:47.389 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:47.386+0000 7f99fe1665c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T16:16:47.389 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:47.386+0000 7f99fe1665c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T16:16:47.390 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:47.387+0000 7f99fe1665c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-11-27T16:16:49.924 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:16:49.929 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:16:50.252 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-11-27T16:16:50.283 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:50.374 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:50.371+0000 7fc3017e05c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-11-27T16:16:50.374 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:50.371+0000 7fc3017e05c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-11-27T16:16:50.375 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:50.371+0000 7fc3017e05c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T16:16:50.375 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:50.371+0000 7fc3017e05c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T16:16:50.375 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:50.371+0000 7fc3017e05c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T16:16:50.375 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:50.372+0000 7fc3017e05c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-11-27T16:16:53.254 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-11-27T16:16:53.286 INFO:tasks.ceph:Reading keys from all nodes... 2022-11-27T16:16:53.287 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:53.288 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-11-27T16:16:53.322 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:53.322 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-11-27T16:16:53.393 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:53.394 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-11-27T16:16:53.462 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:53.462 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-11-27T16:16:53.531 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:53.532 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-11-27T16:16:53.599 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:53.599 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-11-27T16:16:53.666 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:53.667 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-11-27T16:16:53.734 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:53.734 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-11-27T16:16:53.766 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:53.767 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-11-27T16:16:53.837 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:53.837 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2022-11-27T16:16:53.907 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:53.907 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2022-11-27T16:16:53.976 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:53.976 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2022-11-27T16:16:54.044 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:54.045 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2022-11-27T16:16:54.114 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:54.115 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2022-11-27T16:16:54.181 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:54.182 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-11-27T16:16:54.200 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:54.200 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2022-11-27T16:16:54.256 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:54.257 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-11-27T16:16:54.275 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:54.276 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2022-11-27T16:16:54.342 INFO:tasks.ceph:Adding keys to all mons... 2022-11-27T16:16:54.343 DEBUG:teuthology.orchestra.run.smithi134:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T16:16:54.346 DEBUG:teuthology.orchestra.run.smithi191:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T16:16:54.387 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.y] 2022-11-27T16:16:54.388 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBWjYNjFeatHBAA/zg2cdjPhckcKLwkweWKtw== 2022-11-27T16:16:54.388 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2022-11-27T16:16:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBWjYNjxXI+JBAAJ8wzg6Pch2zFvMO7gprTNw== 2022-11-27T16:16:54.389 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2022-11-27T16:16:54.389 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBWjYNj4b6DKxAAIj4BU5ZVyWfXuen2DQ/9Uw== 2022-11-27T16:16:54.390 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.0] 2022-11-27T16:16:54.390 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBajYNjTvFiFxAA8i4ePO8AB/XO63sJbNqfeQ== 2022-11-27T16:16:54.391 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.1] 2022-11-27T16:16:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBdjYNjPgt/FhAAxFx9csU2mLWMVryghzqTPQ== 2022-11-27T16:16:54.392 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.2] 2022-11-27T16:16:54.392 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBgjYNjnJPuFRAA9OH0setAaTrJzZFHjiWO5w== 2022-11-27T16:16:54.393 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.3] 2022-11-27T16:16:54.393 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBjjYNjWn4XFRAAmw7hectGXoPf4zlQWyZ8iQ== 2022-11-27T16:16:54.394 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.x] 2022-11-27T16:16:54.394 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBWjYNj+YjRIBAA+trXfTn9w0YdK17X2454lw== 2022-11-27T16:16:54.395 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2022-11-27T16:16:54.395 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBWjYNjwS2CMBAAG5JMVep5Cpmjztlx5i1uRA== 2022-11-27T16:16:54.396 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2022-11-27T16:16:54.396 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBWjYNjylHZNxAA2op9TXPs+CQeVcSLq+348Q== 2022-11-27T16:16:54.397 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.4] 2022-11-27T16:16:54.397 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBpjYNjat3EGBAAILiotKdwcMWnPK1AR9Oqhw== 2022-11-27T16:16:54.398 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.5] 2022-11-27T16:16:54.398 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBsjYNjxP+kFxAARhdZJwIalp2yxIr7aIQFjQ== 2022-11-27T16:16:54.399 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.6] 2022-11-27T16:16:54.399 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBvjYNj6zMYFxAANX7tFFJPhlKvq1Piqmx9qg== 2022-11-27T16:16:54.400 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.7] 2022-11-27T16:16:54.400 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQByjYNjxME2FhAAm7C1d3MS6+yVTNRAa0FI2A== 2022-11-27T16:16:54.401 INFO:teuthology.orchestra.run.smithi134.stdout:[client.0] 2022-11-27T16:16:54.401 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBXjYNjdVGcABAAGNyB3W6Fw3v+7U1Wh8sYRA== 2022-11-27T16:16:54.402 INFO:teuthology.orchestra.run.smithi134.stdout:[client.2] 2022-11-27T16:16:54.402 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBXjYNjm3thBhAAMsyd5dOV+1RiHVR7PbZIRQ== 2022-11-27T16:16:54.403 INFO:teuthology.orchestra.run.smithi134.stdout:[client.1] 2022-11-27T16:16:54.403 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBXjYNjf13jCRAA7KPJms0+T0hpLPRdpSAa0g== 2022-11-27T16:16:54.403 INFO:teuthology.orchestra.run.smithi134.stdout:[client.3] 2022-11-27T16:16:54.404 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQBXjYNjoCydDxAAB/0Bv/E06iCeva6wHR/eDA== 2022-11-27T16:16:54.408 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.y] 2022-11-27T16:16:54.408 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBWjYNjFeatHBAA/zg2cdjPhckcKLwkweWKtw== 2022-11-27T16:16:54.409 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2022-11-27T16:16:54.409 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBWjYNjxXI+JBAAJ8wzg6Pch2zFvMO7gprTNw== 2022-11-27T16:16:54.409 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2022-11-27T16:16:54.410 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBWjYNj4b6DKxAAIj4BU5ZVyWfXuen2DQ/9Uw== 2022-11-27T16:16:54.410 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.0] 2022-11-27T16:16:54.410 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBajYNjTvFiFxAA8i4ePO8AB/XO63sJbNqfeQ== 2022-11-27T16:16:54.410 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.1] 2022-11-27T16:16:54.410 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBdjYNjPgt/FhAAxFx9csU2mLWMVryghzqTPQ== 2022-11-27T16:16:54.411 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.2] 2022-11-27T16:16:54.411 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBgjYNjnJPuFRAA9OH0setAaTrJzZFHjiWO5w== 2022-11-27T16:16:54.411 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.3] 2022-11-27T16:16:54.411 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBjjYNjWn4XFRAAmw7hectGXoPf4zlQWyZ8iQ== 2022-11-27T16:16:54.412 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.x] 2022-11-27T16:16:54.412 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBWjYNj+YjRIBAA+trXfTn9w0YdK17X2454lw== 2022-11-27T16:16:54.412 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2022-11-27T16:16:54.412 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBWjYNjwS2CMBAAG5JMVep5Cpmjztlx5i1uRA== 2022-11-27T16:16:54.413 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2022-11-27T16:16:54.413 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBWjYNjylHZNxAA2op9TXPs+CQeVcSLq+348Q== 2022-11-27T16:16:54.413 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.4] 2022-11-27T16:16:54.413 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBpjYNjat3EGBAAILiotKdwcMWnPK1AR9Oqhw== 2022-11-27T16:16:54.414 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.5] 2022-11-27T16:16:54.414 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBsjYNjxP+kFxAARhdZJwIalp2yxIr7aIQFjQ== 2022-11-27T16:16:54.414 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.6] 2022-11-27T16:16:54.414 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBvjYNj6zMYFxAANX7tFFJPhlKvq1Piqmx9qg== 2022-11-27T16:16:54.414 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.7] 2022-11-27T16:16:54.415 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQByjYNjxME2FhAAm7C1d3MS6+yVTNRAa0FI2A== 2022-11-27T16:16:54.415 INFO:teuthology.orchestra.run.smithi191.stdout:[client.0] 2022-11-27T16:16:54.415 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBXjYNjdVGcABAAGNyB3W6Fw3v+7U1Wh8sYRA== 2022-11-27T16:16:54.415 INFO:teuthology.orchestra.run.smithi191.stdout:[client.2] 2022-11-27T16:16:54.416 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBXjYNjm3thBhAAMsyd5dOV+1RiHVR7PbZIRQ== 2022-11-27T16:16:54.416 INFO:teuthology.orchestra.run.smithi191.stdout:[client.1] 2022-11-27T16:16:54.416 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBXjYNjf13jCRAA7KPJms0+T0hpLPRdpSAa0g== 2022-11-27T16:16:54.416 INFO:teuthology.orchestra.run.smithi191.stdout:[client.3] 2022-11-27T16:16:54.417 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBXjYNjoCydDxAAB/0Bv/E06iCeva6wHR/eDA== 2022-11-27T16:16:54.418 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:16:54.449 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:16:54.512 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:16:54.514 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:16:54.571 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:16:54.574 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:16:54.629 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:54.633 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:54.687 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:54.690 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:54.743 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:54.746 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:54.798 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:54.801 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:54.854 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:16:54.856 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:16:54.910 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:16:54.913 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:16:54.970 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:16:54.973 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:16:55.030 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:55.033 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:55.086 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:55.088 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:55.142 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:55.145 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:55.198 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:55.201 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:16:55.253 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:16:55.257 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:16:55.351 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:16:55.355 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:16:55.412 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:16:55.415 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:16:55.472 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:16:55.475 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:16:55.532 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-11-27T16:16:55.533 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-11-27T16:16:55.561 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:16:55.659 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-11-27T16:16:55.687 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-11-27T16:16:55.714 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:16:55.811 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-11-27T16:16:55.837 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-11-27T16:16:55.904 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:16:56.002 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-11-27T16:16:56.030 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:56.034 DEBUG:teuthology.orchestra.run.smithi191:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:16:56.088 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-11-27T16:16:56.089 INFO:tasks.ceph.mon.a:Restarting daemon 2022-11-27T16:16:56.090 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-11-27T16:16:56.093 INFO:tasks.ceph.mon.a:Started 2022-11-27T16:16:56.093 INFO:tasks.ceph.mon.b:Restarting daemon 2022-11-27T16:16:56.093 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-11-27T16:16:56.131 INFO:tasks.ceph.mon.b:Started 2022-11-27T16:16:56.131 INFO:tasks.ceph.mon.c:Restarting daemon 2022-11-27T16:16:56.132 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-11-27T16:16:56.134 INFO:tasks.ceph.mon.c:Started 2022-11-27T16:16:56.134 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-11-27T16:16:56.135 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-11-27T16:16:56.135 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-11-27T16:16:56.137 INFO:tasks.ceph.mgr.y:Started 2022-11-27T16:16:56.138 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-11-27T16:16:56.138 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-11-27T16:16:56.140 INFO:tasks.ceph.mgr.x:Started 2022-11-27T16:16:56.141 INFO:tasks.ceph:Setting crush tunables to default 2022-11-27T16:16:56.141 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd crush tunables default 2022-11-27T16:16:56.263 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.261+0000 7fc16c5af700 1 Processor -- start 2022-11-27T16:16:56.263 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.261+0000 7fc16c5af700 1 -- start start 2022-11-27T16:16:56.265 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16c5af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 0x7fc1640b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:56.266 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16c5af700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc1640cdb20 0x7fc16406d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:56.266 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16c5af700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16415c060 con 0x7fc1640cf6a0 2022-11-27T16:16:56.266 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16c5af700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1640af840 con 0x7fc1640cdb20 2022-11-27T16:16:56.266 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16c5af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1641603b0 con 0x7fc1640b19d0 2022-11-27T16:16:56.266 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16ab4c700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc1640cf6a0 0x7fc16406d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:55614/0 (socket says 172.21.15.134:55614) 2022-11-27T16:16:56.267 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16ab4c700 1 -- 172.21.15.134:0/117581867 learned_addr learned my addr 172.21.15.134:0/117581867 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:16:56.267 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16a34b700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 0x7fc1640b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:56.268 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc169b4a700 1 --2- 172.21.15.134:0/117581867 >> v2:172.21.15.191:3300/0 conn(0x7fc1640cdb20 0x7fc16406d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:56.268 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16a34b700 1 -- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 msgr2=0x7fc1640b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T16:16:56.268 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16a34b700 1 -- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 msgr2=0x7fc1640b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:16:56.268 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16a34b700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 0x7fc1640b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:16:56.269 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc16a34b700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 0x7fc1640b1da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T16:16:56.269 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc169b4a700 1 -- 172.21.15.134:0/117581867 >> v2:172.21.15.191:3300/0 conn(0x7fc1640cdb20 msgr2=0x7fc16406d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T16:16:56.269 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc169b4a700 1 -- 172.21.15.134:0/117581867 >> v2:172.21.15.191:3300/0 conn(0x7fc1640cdb20 msgr2=0x7fc16406d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:16:56.269 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc169b4a700 1 --2- 172.21.15.134:0/117581867 >> v2:172.21.15.191:3300/0 conn(0x7fc1640cdb20 0x7fc16406d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:16:56.270 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.262+0000 7fc169b4a700 1 --2- 172.21.15.134:0/117581867 >> v2:172.21.15.191:3300/0 conn(0x7fc1640cdb20 0x7fc16406d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T16:16:56.436 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.434+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403972585 0 0) 0x7fc154001490 con 0x7fc1640cf6a0 2022-11-27T16:16:56.436 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.434+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1500035b0 con 0x7fc1640cf6a0 2022-11-27T16:16:56.440 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3316764963 0 0) 0x7fc1500035b0 con 0x7fc1640cf6a0 2022-11-27T16:16:56.441 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 msgr2=0x7fc1640b1da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T16:16:56.441 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 0x7fc1640b1da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:56.441 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 >> v2:172.21.15.191:3300/0 conn(0x7fc1640cdb20 msgr2=0x7fc16406d060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T16:16:56.441 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 --2- 172.21.15.134:0/117581867 >> v2:172.21.15.191:3300/0 conn(0x7fc1640cdb20 0x7fc16406d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:56.442 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1640b22e0 con 0x7fc1640cf6a0 2022-11-27T16:16:56.442 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2726658286 0 0) 0x7fc154001b80 con 0x7fc1640cf6a0 2022-11-27T16:16:56.442 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 >> v1:172.21.15.134:6789/0 conn(0x7fc1640cf6a0 legacy=0x7fc16406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:56.442 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc150003a00 0x7fc150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:56.442 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc150004850 0x7fc150004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:56.443 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 0x7fc150005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:56.443 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640af840 con 0x7fc150004850 2022-11-27T16:16:56.443 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1641603b0 con 0x7fc150003a00 2022-11-27T16:16:56.443 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15000bb10 con 0x7fc1640b19d0 2022-11-27T16:16:56.444 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169b4a700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc150004850 0x7fc150004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:56.444 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc16a34b700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc150003a00 0x7fc150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:56.444 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc16ab4c700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 0x7fc150005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:56.444 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169b4a700 1 -- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 msgr2=0x7fc150005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:16:56.446 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169b4a700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1640b19d0 0x7fc150005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:56.446 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169b4a700 1 -- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc150003a00 msgr2=0x7fc150003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:16:56.446 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169b4a700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc150003a00 0x7fc150003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:56.446 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169b4a700 1 -- 172.21.15.134:0/117581867 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1500035b0 con 0x7fc150004850 2022-11-27T16:16:56.447 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.436+0000 7fc169b4a700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc150004850 0x7fc150004cc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc158009890 tx=0x7fc150005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd017c85055ec72 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:56.447 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.437+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15800c070 con 0x7fc150004850 2022-11-27T16:16:56.447 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.437+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc158004020 con 0x7fc150004850 2022-11-27T16:16:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.437+0000 7fc169349700 1 -- 172.21.15.134:0/117581867 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1580043b0 con 0x7fc150004850 2022-11-27T16:16:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.437+0000 7fc16c5af700 1 -- 172.21.15.134:0/117581867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc150004850 msgr2=0x7fc150004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.437+0000 7fc16c5af700 1 --2- 172.21.15.134:0/117581867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc150004850 0x7fc150004cc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc158009890 tx=0x7fc150005880 comp rx=0 tx=0).stop 2022-11-27T16:16:56.448 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.437+0000 7fc16ab4c700 1 -- 172.21.15.134:0/117581867 reap_dead start 2022-11-27T16:16:56.449 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.437+0000 7fc16c5af700 1 -- 172.21.15.134:0/117581867 shutdown_connections 2022-11-27T16:16:56.449 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.437+0000 7fc16c5af700 1 -- 172.21.15.134:0/117581867 >> 172.21.15.134:0/117581867 conn(0x7fc164069d10 msgr2=0x7fc164099270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:56.449 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.437+0000 7fc16c5af700 1 -- 172.21.15.134:0/117581867 shutdown_connections 2022-11-27T16:16:56.450 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.438+0000 7fc16c5af700 1 -- 172.21.15.134:0/117581867 wait complete. 2022-11-27T16:16:56.450 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.439+0000 7fc16c5af700 1 Processor -- start 2022-11-27T16:16:56.450 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.439+0000 7fc16c5af700 1 -- start start 2022-11-27T16:16:56.453 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc16c5af700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc150004850 0x7fc1640b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:56.453 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc16c5af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16415c200 0x7fc164150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:56.453 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc16c5af700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16415d9d0 0x7fc1640adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:56.454 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc16c5af700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc164150ac0 con 0x7fc150004850 2022-11-27T16:16:56.454 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc16c5af700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16415c670 con 0x7fc16415d9d0 2022-11-27T16:16:56.454 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc16c5af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16415c930 con 0x7fc16415c200 2022-11-27T16:16:56.454 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc16ab4c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16415d9d0 0x7fc1640adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:56.454 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc169b4a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16415c200 0x7fc164150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:56.455 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc169b4a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16415c200 0x7fc164150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.134:41356/0 (socket says 172.21.15.134:41356) 2022-11-27T16:16:56.455 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc16ab4c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16415d9d0 0x7fc1640adc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.134:53970/0 (socket says 172.21.15.134:53970) 2022-11-27T16:16:56.455 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc169b4a700 1 -- 172.21.15.134:0/4220227590 learned_addr learned my addr 172.21.15.134:0/4220227590 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:16:56.455 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc169b4a700 1 -- 172.21.15.134:0/4220227590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16415d9d0 msgr2=0x7fc1640adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:56.456 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc169b4a700 1 --2- 172.21.15.134:0/4220227590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16415d9d0 0x7fc1640adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:56.456 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc169b4a700 1 -- 172.21.15.134:0/4220227590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc150004850 msgr2=0x7fc1640b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:56.456 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc169b4a700 1 --2- 172.21.15.134:0/4220227590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc150004850 0x7fc1640b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:56.456 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc169b4a700 1 -- 172.21.15.134:0/4220227590 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16415cc10 con 0x7fc16415c200 2022-11-27T16:16:56.457 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.440+0000 7fc169b4a700 1 --2- 172.21.15.134:0/4220227590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16415c200 0x7fc164150580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc158009fd0 tx=0x7fc1580047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:56.457 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.442+0000 7fc162ffd700 1 -- 172.21.15.134:0/4220227590 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1580038b0 con 0x7fc16415c200 2022-11-27T16:16:56.457 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.442+0000 7fc162ffd700 1 -- 172.21.15.134:0/4220227590 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc158003a10 con 0x7fc16415c200 2022-11-27T16:16:56.457 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.442+0000 7fc16c5af700 1 -- 172.21.15.134:0/4220227590 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16415cef0 con 0x7fc16415c200 2022-11-27T16:16:56.458 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.442+0000 7fc162ffd700 1 -- 172.21.15.134:0/4220227590 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158003dc0 con 0x7fc16415c200 2022-11-27T16:16:56.458 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.442+0000 7fc16c5af700 1 -- 172.21.15.134:0/4220227590 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16415d4b0 con 0x7fc16415c200 2022-11-27T16:16:56.458 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.442+0000 7fc162ffd700 1 -- 172.21.15.134:0/4220227590 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc15802d020 con 0x7fc16415c200 2022-11-27T16:16:56.458 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.442+0000 7fc162ffd700 1 -- 172.21.15.134:0/4220227590 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc158030440 con 0x7fc16415c200 2022-11-27T16:16:56.458 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.443+0000 7fc16c5af700 1 -- 172.21.15.134:0/4220227590 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16415d670 con 0x7fc16415c200 2022-11-27T16:16:56.459 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.446+0000 7fc162ffd700 1 -- 172.21.15.134:0/4220227590 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fc158017030 con 0x7fc16415c200 2022-11-27T16:16:56.536 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.535+0000 7fc16c5af700 1 -- 172.21.15.134:0/4220227590 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fc16415d820 con 0x7fc16415c200 2022-11-27T16:16:56.544 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.543+0000 7fc162ffd700 1 -- 172.21.15.134:0/4220227590 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc15800f040 con 0x7fc16415c200 2022-11-27T16:16:56.546 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.544+0000 7fc16c5af700 1 -- 172.21.15.134:0/4220227590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16415c200 msgr2=0x7fc164150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:56.546 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.544+0000 7fc16c5af700 1 --2- 172.21.15.134:0/4220227590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16415c200 0x7fc164150580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc158009fd0 tx=0x7fc1580047c0 comp rx=0 tx=0).stop 2022-11-27T16:16:56.547 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.544+0000 7fc16c5af700 1 -- 172.21.15.134:0/4220227590 shutdown_connections 2022-11-27T16:16:56.547 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.544+0000 7fc16c5af700 1 --2- 172.21.15.134:0/4220227590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16415d9d0 0x7fc1640adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:56.547 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.544+0000 7fc16c5af700 1 --2- 172.21.15.134:0/4220227590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc150004850 0x7fc1640b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:56.547 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.544+0000 7fc16c5af700 1 --2- 172.21.15.134:0/4220227590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16415c200 0x7fc164150580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.544+0000 7fc16c5af700 1 -- 172.21.15.134:0/4220227590 >> 172.21.15.134:0/4220227590 conn(0x7fc164069d10 msgr2=0x7fc16415b530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.544+0000 7fc16c5af700 1 -- 172.21.15.134:0/4220227590 shutdown_connections 2022-11-27T16:16:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:56.544+0000 7fc16c5af700 1 -- 172.21.15.134:0/4220227590 wait complete. 2022-11-27T16:16:56.549 INFO:teuthology.orchestra.run.smithi134.stderr:adjusted tunables profile to default 2022-11-27T16:16:56.550 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:56.547+0000 7f46006a3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T16:16:56.562 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-11-27T16:16:56.562 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:56.563 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-11-27T16:16:56.568 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:56.567+0000 7f75e0047200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T16:16:56.589 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:56.590 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-11-27T16:16:56.612 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:56.610+0000 7f46006a3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T16:16:56.636 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:56.635+0000 7f75e0047200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T16:16:56.658 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:56.658 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-11-27T16:16:56.685 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:56.683+0000 7f46006a3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T16:16:56.716 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:56.714+0000 7f75e0047200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T16:16:56.723 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2022-11-27T16:16:56.724 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-11-27T16:16:56.762 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:56.760+0000 7f46006a3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T16:16:56.790 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:56.791 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2022-11-27T16:16:56.799 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:56.798+0000 7f75e0047200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T16:16:56.818 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:56.819 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2022-11-27T16:16:56.890 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:56.890 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2022-11-27T16:16:56.906 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:56.904+0000 7f46006a3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T16:16:56.932 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-11-27T16:16:56.932 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2022-11-27T16:16:56.939 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:56.938+0000 7f75e0047200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T16:16:56.998 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new f44064b8-6282-45ff-94ab-f3a2eb231dc1 0 2022-11-27T16:16:57.156 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.154+0000 7f9eda195700 1 Processor -- start 2022-11-27T16:16:57.156 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.154+0000 7f9eda195700 1 -- start start 2022-11-27T16:16:57.160 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.155+0000 7f9eda195700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9ed40b2dc0 0x7f9ed40b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.161 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.155+0000 7f9eda195700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed40b1ca0 con 0x7f9ed406d120 2022-11-27T16:16:57.161 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.155+0000 7f9eda195700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed40d0cb0 con 0x7f9ed41598c0 2022-11-27T16:16:57.162 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.155+0000 7f9eda195700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed40cc830 con 0x7f9ed40b2dc0 2022-11-27T16:16:57.162 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.155+0000 7f9ed37fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f9ed41598c0 0x7f9ed40d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:44640/0 (socket says 172.21.15.191:44640) 2022-11-27T16:16:57.163 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.155+0000 7f9ed37fe700 1 -- 172.21.15.191:0/1071034018 learned_addr learned my addr 172.21.15.191:0/1071034018 (peer_addr_for_me v1:172.21.15.191:0/0) 2022-11-27T16:16:57.163 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed2ffd700 1 --2- 172.21.15.191:0/1071034018 >> v2:172.21.15.134:3300/0 conn(0x7f9ed40b2dc0 0x7f9ed40b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.163 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed27fc700 1 -- 172.21.15.191:0/1071034018 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452962872 0 0) 0x7f9ed40d0cb0 con 0x7f9ed41598c0 2022-11-27T16:16:57.164 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed27fc700 1 -- 172.21.15.191:0/1071034018 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec00035b0 con 0x7f9ed41598c0 2022-11-27T16:16:57.164 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed27fc700 1 -- 172.21.15.191:0/1071034018 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304020222 0 0) 0x7f9ed40b1ca0 con 0x7f9ed406d120 2022-11-27T16:16:57.164 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed27fc700 1 -- 172.21.15.191:0/1071034018 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed40d0cb0 con 0x7f9ed406d120 2022-11-27T16:16:57.165 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed2ffd700 1 -- 172.21.15.191:0/1071034018 >> v1:172.21.15.191:6789/0 conn(0x7f9ed41598c0 legacy=0x7f9ed40d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.165 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed2ffd700 1 -- 172.21.15.191:0/1071034018 >> v1:172.21.15.134:6789/0 conn(0x7f9ed406d120 legacy=0x7f9ed40cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.166 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed2ffd700 1 -- 172.21.15.191:0/1071034018 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40cc270 con 0x7f9ed40b2dc0 2022-11-27T16:16:57.167 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed2ffd700 1 --2- 172.21.15.191:0/1071034018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed40b2dc0 0x7f9ed40b3230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009840 tx=0x7f9ec8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162b07ad6d5e590b server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:57.167 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed27fc700 1 -- 172.21.15.191:0/1071034018 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8011830 con 0x7f9ed40b2dc0 2022-11-27T16:16:57.167 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed27fc700 1 -- 172.21.15.191:0/1071034018 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec8011990 con 0x7f9ed40b2dc0 2022-11-27T16:16:57.168 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.156+0000 7f9ed27fc700 1 -- 172.21.15.191:0/1071034018 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8011d20 con 0x7f9ed40b2dc0 2022-11-27T16:16:57.168 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.157+0000 7f9eda195700 1 -- 172.21.15.191:0/1071034018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed40b2dc0 msgr2=0x7f9ed40b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.168 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.157+0000 7f9eda195700 1 --2- 172.21.15.191:0/1071034018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed40b2dc0 0x7f9ed40b3230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009840 tx=0x7f9ec8009b50 comp rx=0 tx=0).stop 2022-11-27T16:16:57.168 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.157+0000 7f9eda195700 1 -- 172.21.15.191:0/1071034018 shutdown_connections 2022-11-27T16:16:57.169 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.157+0000 7f9eda195700 1 --2- 172.21.15.191:0/1071034018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed40b2dc0 0x7f9ed40b3230 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.169 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.157+0000 7f9eda195700 1 -- 172.21.15.191:0/1071034018 >> 172.21.15.191:0/1071034018 conn(0x7f9ed4071060 msgr2=0x7f9ed4071430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:57.169 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.157+0000 7f9eda195700 1 -- 172.21.15.191:0/1071034018 shutdown_connections 2022-11-27T16:16:57.169 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.157+0000 7f9eda195700 1 -- 172.21.15.191:0/1071034018 wait complete. 2022-11-27T16:16:57.173 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.159+0000 7f9eda195700 1 Processor -- start 2022-11-27T16:16:57.173 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.159+0000 7f9eda195700 1 -- start start 2022-11-27T16:16:57.173 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.160+0000 7f9eda195700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed406d120 0x7f9ed40b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.173 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.160+0000 7f9eda195700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ed40b2dc0 0x7f9ed40c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.174 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9eda195700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ed41598c0 0x7f9ed40c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.174 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9eda195700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40c9950 con 0x7f9ed406d120 2022-11-27T16:16:57.174 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9eda195700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40c83e0 con 0x7f9ed40b2dc0 2022-11-27T16:16:57.175 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9eda195700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40c7a10 con 0x7f9ed41598c0 2022-11-27T16:16:57.175 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed2ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ed40b2dc0 0x7f9ed40c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.176 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ed41598c0 0x7f9ed40c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.176 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed406d120 0x7f9ed40b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.176 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed2ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ed40b2dc0 0x7f9ed40c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45420/0 (socket says 172.21.15.191:45420) 2022-11-27T16:16:57.176 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed2ffd700 1 -- 172.21.15.191:0/1592402881 learned_addr learned my addr 172.21.15.191:0/1592402881 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:57.177 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed3fff700 1 -- 172.21.15.191:0/1592402881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ed40b2dc0 msgr2=0x7f9ed40c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.177 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed3fff700 1 --2- 172.21.15.191:0/1592402881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ed40b2dc0 0x7f9ed40c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.177 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed3fff700 1 -- 172.21.15.191:0/1592402881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed406d120 msgr2=0x7f9ed40b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.177 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed3fff700 1 --2- 172.21.15.191:0/1592402881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed406d120 0x7f9ed40b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.178 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed3fff700 1 -- 172.21.15.191:0/1592402881 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80094a0 con 0x7f9ed41598c0 2022-11-27T16:16:57.178 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed2ffd700 1 --2- 172.21.15.191:0/1592402881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ed40b2dc0 0x7f9ed40c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:57.178 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed37fe700 1 --2- 172.21.15.191:0/1592402881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed406d120 0x7f9ed40b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:57.178 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ed3fff700 1 --2- 172.21.15.191:0/1592402881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ed41598c0 0x7f9ed40c7ea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00cbf0 tx=0x7f9ecc00cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:57.178 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ebbfff700 1 -- 172.21.15.191:0/1592402881 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc003c00 con 0x7f9ed41598c0 2022-11-27T16:16:57.179 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ebbfff700 1 -- 172.21.15.191:0/1592402881 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc003d60 con 0x7f9ed41598c0 2022-11-27T16:16:57.179 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9eda195700 1 -- 172.21.15.191:0/1592402881 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed40c9b60 con 0x7f9ed41598c0 2022-11-27T16:16:57.179 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ebbfff700 1 -- 172.21.15.191:0/1592402881 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc00aa40 con 0x7f9ed41598c0 2022-11-27T16:16:57.179 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9eda195700 1 -- 172.21.15.191:0/1592402881 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed40c73c0 con 0x7f9ed41598c0 2022-11-27T16:16:57.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.161+0000 7f9ebbfff700 1 -- 172.21.15.191:0/1592402881 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9ecc010050 con 0x7f9ed41598c0 2022-11-27T16:16:57.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.162+0000 7f9ebbfff700 1 -- 172.21.15.191:0/1592402881 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9ecc012760 con 0x7f9ed41598c0 2022-11-27T16:16:57.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.162+0000 7f9eda195700 1 -- 172.21.15.191:0/1592402881 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed40c7b80 con 0x7f9ed41598c0 2022-11-27T16:16:57.181 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.164+0000 7f9ebbfff700 1 -- 172.21.15.191:0/1592402881 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f9ecc0050d0 con 0x7f9ed41598c0 2022-11-27T16:16:57.259 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:57.257+0000 7f46006a3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T16:16:57.262 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.260+0000 7f9eda195700 1 -- 172.21.15.191:0/1592402881 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f44064b8-6282-45ff-94ab-f3a2eb231dc1", "id": 0} v 0) v1 -- 0x7f9ed4073030 con 0x7f9ed41598c0 2022-11-27T16:16:57.267 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:57.265+0000 7f75e0047200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T16:16:57.267 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.266+0000 7f9ebbfff700 1 -- 172.21.15.191:0/1592402881 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f44064b8-6282-45ff-94ab-f3a2eb231dc1", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9ecc005270 con 0x7f9ed41598c0 2022-11-27T16:16:57.268 INFO:teuthology.orchestra.run.smithi191.stdout:0 2022-11-27T16:16:57.269 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.267+0000 7f9eda195700 1 -- 172.21.15.191:0/1592402881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ed41598c0 msgr2=0x7f9ed40c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.269 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.267+0000 7f9eda195700 1 --2- 172.21.15.191:0/1592402881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ed41598c0 0x7f9ed40c7ea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00cbf0 tx=0x7f9ecc00cf00 comp rx=0 tx=0).stop 2022-11-27T16:16:57.269 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.267+0000 7f9eda195700 1 -- 172.21.15.191:0/1592402881 shutdown_connections 2022-11-27T16:16:57.270 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.267+0000 7f9eda195700 1 --2- 172.21.15.191:0/1592402881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ed41598c0 0x7f9ed40c7ea0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.270 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.267+0000 7f9eda195700 1 --2- 172.21.15.191:0/1592402881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed406d120 0x7f9ed40b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.270 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.267+0000 7f9eda195700 1 --2- 172.21.15.191:0/1592402881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ed40b2dc0 0x7f9ed40c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.270 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.267+0000 7f9eda195700 1 -- 172.21.15.191:0/1592402881 >> 172.21.15.191:0/1592402881 conn(0x7f9ed4071060 msgr2=0x7f9ed4157ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:57.271 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.267+0000 7f9eda195700 1 -- 172.21.15.191:0/1592402881 shutdown_connections 2022-11-27T16:16:57.271 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.267+0000 7f9eda195700 1 -- 172.21.15.191:0/1592402881 wait complete. 2022-11-27T16:16:57.281 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new af1b8ac9-0335-4550-aaf8-9f15545625e8 1 2022-11-27T16:16:57.338 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:57.336+0000 7f46006a3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T16:16:57.340 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:57.338+0000 7f75e0047200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T16:16:57.403 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.401+0000 7fb0eafad700 1 Processor -- start 2022-11-27T16:16:57.403 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.401+0000 7fb0eafad700 1 -- start start 2022-11-27T16:16:57.406 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0eafad700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e406ff80 0x7fb0e4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.407 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0eafad700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb0e40ce510 0x7fb0e406c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.407 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0eafad700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e4070890 con 0x7fb0e40c4620 2022-11-27T16:16:57.407 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0eafad700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e40c4b00 con 0x7fb0e40ce510 2022-11-27T16:16:57.407 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0eafad700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e40cda30 con 0x7fb0e406ff80 2022-11-27T16:16:57.408 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e8d49700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e406ff80 0x7fb0e4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.408 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e954a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb0e40ce510 0x7fb0e406c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.408 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e8d49700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e406ff80 0x7fb0e4070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:54186/0 (socket says 172.21.15.191:54186) 2022-11-27T16:16:57.408 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e8d49700 1 -- 172.21.15.191:0/4026911546 learned_addr learned my addr 172.21.15.191:0/4026911546 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:57.408 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e954a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb0e40ce510 0x7fb0e406c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45428/0 (socket says 172.21.15.191:45428) 2022-11-27T16:16:57.409 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e8d49700 1 -- 172.21.15.191:0/4026911546 >> v2:172.21.15.191:3300/0 conn(0x7fb0e40ce510 msgr2=0x7fb0e406c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:16:57.409 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e8d49700 1 --2- 172.21.15.191:0/4026911546 >> v2:172.21.15.191:3300/0 conn(0x7fb0e40ce510 0x7fb0e406c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.409 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e8d49700 1 -- 172.21.15.191:0/4026911546 >> v1:172.21.15.134:6789/0 conn(0x7fb0e40c4620 legacy=0x7fb0e40c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.409 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e8d49700 1 -- 172.21.15.191:0/4026911546 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e40cdd10 con 0x7fb0e406ff80 2022-11-27T16:16:57.410 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e954a700 1 --2- 172.21.15.191:0/4026911546 >> v2:172.21.15.191:3300/0 conn(0x7fb0e40ce510 0x7fb0e406c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:57.410 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.402+0000 7fb0e8d49700 1 --2- 172.21.15.191:0/4026911546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e406ff80 0x7fb0e4070350 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80098f0 tx=0x7fb0d8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb5550764ed9b9b2 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:57.410 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0e37fe700 1 -- 172.21.15.191:0/4026911546 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8011830 con 0x7fb0e406ff80 2022-11-27T16:16:57.410 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0e37fe700 1 -- 172.21.15.191:0/4026911546 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d8011990 con 0x7fb0e406ff80 2022-11-27T16:16:57.410 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0e37fe700 1 -- 172.21.15.191:0/4026911546 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8011d20 con 0x7fb0e406ff80 2022-11-27T16:16:57.411 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0eafad700 1 -- 172.21.15.191:0/4026911546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e406ff80 msgr2=0x7fb0e4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.411 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0eafad700 1 --2- 172.21.15.191:0/4026911546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e406ff80 0x7fb0e4070350 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80098f0 tx=0x7fb0d8009c00 comp rx=0 tx=0).stop 2022-11-27T16:16:57.411 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0eafad700 1 -- 172.21.15.191:0/4026911546 shutdown_connections 2022-11-27T16:16:57.411 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0eafad700 1 --2- 172.21.15.191:0/4026911546 >> v2:172.21.15.191:3300/0 conn(0x7fb0e40ce510 0x7fb0e406c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.412 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0eafad700 1 --2- 172.21.15.191:0/4026911546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e406ff80 0x7fb0e4070350 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.412 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0eafad700 1 -- 172.21.15.191:0/4026911546 >> 172.21.15.191:0/4026911546 conn(0x7fb0e40b16c0 msgr2=0x7fb0e40cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:57.412 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0eafad700 1 -- 172.21.15.191:0/4026911546 shutdown_connections 2022-11-27T16:16:57.412 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.403+0000 7fb0eafad700 1 -- 172.21.15.191:0/4026911546 wait complete. 2022-11-27T16:16:57.412 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.405+0000 7fb0eafad700 1 Processor -- start 2022-11-27T16:16:57.413 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.405+0000 7fb0eafad700 1 -- start start 2022-11-27T16:16:57.415 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.405+0000 7fb0eafad700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e406ff80 0x7fb0e4149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.416 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.405+0000 7fb0eafad700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e40c4620 0x7fb0e4156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.416 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.405+0000 7fb0eafad700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e40ce510 0x7fb0e40c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.416 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.405+0000 7fb0eafad700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e4149690 con 0x7fb0e40ce510 2022-11-27T16:16:57.416 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.405+0000 7fb0eafad700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e4154c70 con 0x7fb0e406ff80 2022-11-27T16:16:57.416 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.405+0000 7fb0eafad700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e41ef720 con 0x7fb0e40c4620 2022-11-27T16:16:57.417 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e8d49700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e406ff80 0x7fb0e4149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.417 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e40c4620 0x7fb0e4156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.417 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e8d49700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e406ff80 0x7fb0e4149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45430/0 (socket says 172.21.15.191:45430) 2022-11-27T16:16:57.417 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e8d49700 1 -- 172.21.15.191:0/1447573284 learned_addr learned my addr 172.21.15.191:0/1447573284 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:57.418 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e954a700 1 --2- 172.21.15.191:0/1447573284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e40ce510 0x7fb0e40c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.418 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e8d49700 1 -- 172.21.15.191:0/1447573284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e40c4620 msgr2=0x7fb0e4156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.418 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e8d49700 1 --2- 172.21.15.191:0/1447573284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e40c4620 0x7fb0e4156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.418 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e8d49700 1 -- 172.21.15.191:0/1447573284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e40ce510 msgr2=0x7fb0e40c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.418 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e8d49700 1 --2- 172.21.15.191:0/1447573284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e40ce510 0x7fb0e40c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.419 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e8d49700 1 -- 172.21.15.191:0/1447573284 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc009a70 con 0x7fb0e406ff80 2022-11-27T16:16:57.419 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e3fff700 1 --2- 172.21.15.191:0/1447573284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e40c4620 0x7fb0e4156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:57.419 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e8d49700 1 --2- 172.21.15.191:0/1447573284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e406ff80 0x7fb0e4149150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8000c00 tx=0x7fb0d8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:57.419 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e17fa700 1 -- 172.21.15.191:0/1447573284 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80042c0 con 0x7fb0e406ff80 2022-11-27T16:16:57.420 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e17fa700 1 -- 172.21.15.191:0/1447573284 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d801f070 con 0x7fb0e406ff80 2022-11-27T16:16:57.420 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0eafad700 1 -- 172.21.15.191:0/1447573284 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d8009550 con 0x7fb0e406ff80 2022-11-27T16:16:57.420 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0e17fa700 1 -- 172.21.15.191:0/1447573284 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80036a0 con 0x7fb0e406ff80 2022-11-27T16:16:57.420 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.406+0000 7fb0eafad700 1 -- 172.21.15.191:0/1447573284 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e4155160 con 0x7fb0e406ff80 2022-11-27T16:16:57.420 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.407+0000 7fb0e17fa700 1 -- 172.21.15.191:0/1447573284 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fb0d802a020 con 0x7fb0e406ff80 2022-11-27T16:16:57.421 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.407+0000 7fb0e17fa700 1 -- 172.21.15.191:0/1447573284 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fb0d801a520 con 0x7fb0e406ff80 2022-11-27T16:16:57.421 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.407+0000 7fb0eafad700 1 -- 172.21.15.191:0/1447573284 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e41497e0 con 0x7fb0e406ff80 2022-11-27T16:16:57.421 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.409+0000 7fb0e17fa700 1 -- 172.21.15.191:0/1447573284 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fb0d8015020 con 0x7fb0e406ff80 2022-11-27T16:16:57.465 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:57.463+0000 7f46006a3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T16:16:57.467 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:57.466+0000 7f75e0047200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T16:16:57.495 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.494+0000 7fb0eafad700 1 -- 172.21.15.191:0/1447573284 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "af1b8ac9-0335-4550-aaf8-9f15545625e8", "id": 1} v 0) v1 -- 0x7fb0e4065ac0 con 0x7fb0e406ff80 2022-11-27T16:16:57.503 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.501+0000 7fb0e17fa700 1 -- 172.21.15.191:0/1447573284 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af1b8ac9-0335-4550-aaf8-9f15545625e8", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb0d80038d0 con 0x7fb0e406ff80 2022-11-27T16:16:57.503 INFO:teuthology.orchestra.run.smithi191.stdout:1 2022-11-27T16:16:57.505 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.502+0000 7fb0eafad700 1 -- 172.21.15.191:0/1447573284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e406ff80 msgr2=0x7fb0e4149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.505 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.502+0000 7fb0eafad700 1 --2- 172.21.15.191:0/1447573284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e406ff80 0x7fb0e4149150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8000c00 tx=0x7fb0d8004750 comp rx=0 tx=0).stop 2022-11-27T16:16:57.505 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.502+0000 7fb0eafad700 1 -- 172.21.15.191:0/1447573284 shutdown_connections 2022-11-27T16:16:57.505 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.502+0000 7fb0eafad700 1 --2- 172.21.15.191:0/1447573284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e40ce510 0x7fb0e40c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.506 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.502+0000 7fb0eafad700 1 --2- 172.21.15.191:0/1447573284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e406ff80 0x7fb0e4149150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.506 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.502+0000 7fb0eafad700 1 --2- 172.21.15.191:0/1447573284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e40c4620 0x7fb0e4156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.506 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.502+0000 7fb0eafad700 1 -- 172.21.15.191:0/1447573284 >> 172.21.15.191:0/1447573284 conn(0x7fb0e40b16c0 msgr2=0x7fb0e406d590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:57.506 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.502+0000 7fb0eafad700 1 -- 172.21.15.191:0/1447573284 shutdown_connections 2022-11-27T16:16:57.506 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.502+0000 7fb0eafad700 1 -- 172.21.15.191:0/1447573284 wait complete. 2022-11-27T16:16:57.516 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 5bbdd443-3ec5-40fb-86af-3e5a6ab1606a 2 2022-11-27T16:16:57.546 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:57.545+0000 7f75e0047200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T16:16:57.561 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:57.559+0000 7f46006a3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T16:16:57.644 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.633+0000 7f6fd586b700 1 Processor -- start 2022-11-27T16:16:57.644 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.633+0000 7f6fd586b700 1 -- start start 2022-11-27T16:16:57.644 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.634+0000 7f6fd586b700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6fd015ba40 0x7f6fd00ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.645 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.634+0000 7f6fd586b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6fd00b0760 0x7f6fd00693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.645 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.634+0000 7f6fd586b700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd00ce050 con 0x7f6fd00cd600 2022-11-27T16:16:57.646 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.634+0000 7f6fd586b700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00cc240 con 0x7f6fd015ba40 2022-11-27T16:16:57.646 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.634+0000 7f6fd586b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00cc520 con 0x7f6fd00b0760 2022-11-27T16:16:57.647 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.634+0000 7f6fcf7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6fd00b0760 0x7f6fd00693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.647 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.634+0000 7f6fcf7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6fd00b0760 0x7f6fd00693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45440/0 (socket says 172.21.15.191:45440) 2022-11-27T16:16:57.647 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.634+0000 7f6fcf7fe700 1 -- 172.21.15.191:0/3943871571 learned_addr learned my addr 172.21.15.191:0/3943871571 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:57.648 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.634+0000 7f6fceffd700 1 --2- 172.21.15.191:0/3943871571 >> v2:172.21.15.134:3300/0 conn(0x7f6fd015ba40 0x7f6fd00ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.648 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fcf7fe700 1 -- 172.21.15.191:0/3943871571 >> v2:172.21.15.134:3300/0 conn(0x7f6fd015ba40 msgr2=0x7f6fd00ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:16:57.648 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fcf7fe700 1 --2- 172.21.15.191:0/3943871571 >> v2:172.21.15.134:3300/0 conn(0x7f6fd015ba40 0x7f6fd00ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.649 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fcf7fe700 1 -- 172.21.15.191:0/3943871571 >> v1:172.21.15.134:6789/0 conn(0x7f6fd00cd600 legacy=0x7f6fd00b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.649 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fcf7fe700 1 -- 172.21.15.191:0/3943871571 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00cc800 con 0x7f6fd00b0760 2022-11-27T16:16:57.650 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fceffd700 1 --2- 172.21.15.191:0/3943871571 >> v2:172.21.15.134:3300/0 conn(0x7f6fd015ba40 0x7f6fd00ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:16:57.650 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fcf7fe700 1 --2- 172.21.15.191:0/3943871571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 0x7f6fd00693f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009e50 tx=0x7f6fc800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10bca42d5e8c172 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:57.650 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fcdffb700 1 -- 172.21.15.191:0/3943871571 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800e070 con 0x7f6fd00b0760 2022-11-27T16:16:57.651 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fcdffb700 1 -- 172.21.15.191:0/3943871571 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc800b980 con 0x7f6fd00b0760 2022-11-27T16:16:57.651 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fcdffb700 1 -- 172.21.15.191:0/3943871571 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800bd30 con 0x7f6fd00b0760 2022-11-27T16:16:57.651 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fd586b700 1 -- 172.21.15.191:0/3943871571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 msgr2=0x7f6fd00693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.652 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fd586b700 1 --2- 172.21.15.191:0/3943871571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 0x7f6fd00693f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009e50 tx=0x7f6fc800b330 comp rx=0 tx=0).stop 2022-11-27T16:16:57.652 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fd586b700 1 -- 172.21.15.191:0/3943871571 shutdown_connections 2022-11-27T16:16:57.652 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fd586b700 1 --2- 172.21.15.191:0/3943871571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 0x7f6fd00693f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.653 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fd586b700 1 --2- 172.21.15.191:0/3943871571 >> v2:172.21.15.134:3300/0 conn(0x7f6fd015ba40 0x7f6fd00ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.653 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fd586b700 1 -- 172.21.15.191:0/3943871571 >> 172.21.15.191:0/3943871571 conn(0x7f6fd00cb050 msgr2=0x7f6fd00cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:57.653 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.635+0000 7f6fd586b700 1 -- 172.21.15.191:0/3943871571 shutdown_connections 2022-11-27T16:16:57.653 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.636+0000 7f6fd586b700 1 -- 172.21.15.191:0/3943871571 wait complete. 2022-11-27T16:16:57.653 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.637+0000 7f6fd586b700 1 Processor -- start 2022-11-27T16:16:57.654 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.637+0000 7f6fd586b700 1 -- start start 2022-11-27T16:16:57.654 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.637+0000 7f6fd586b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 0x7f6fd01f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.654 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fd586b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fd00cd600 0x7f6fd00b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.654 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fd586b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd015ba40 0x7f6fd00c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.654 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fd586b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00c8060 con 0x7f6fd015ba40 2022-11-27T16:16:57.655 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fd586b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd01f5770 con 0x7f6fd00b0760 2022-11-27T16:16:57.655 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fd586b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00ae340 con 0x7f6fd00cd600 2022-11-27T16:16:57.655 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fceffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 0x7f6fd01f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.655 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fce7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fd00cd600 0x7f6fd00b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.656 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fceffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 0x7f6fd01f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45442/0 (socket says 172.21.15.191:45442) 2022-11-27T16:16:57.656 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fceffd700 1 -- 172.21.15.191:0/3891346906 learned_addr learned my addr 172.21.15.191:0/3891346906 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:57.656 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fcf7fe700 1 --2- 172.21.15.191:0/3891346906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd015ba40 0x7f6fd00c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.656 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fceffd700 1 -- 172.21.15.191:0/3891346906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fd00cd600 msgr2=0x7f6fd00b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.657 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fceffd700 1 --2- 172.21.15.191:0/3891346906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fd00cd600 0x7f6fd00b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.657 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fceffd700 1 -- 172.21.15.191:0/3891346906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd015ba40 msgr2=0x7f6fd00c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.657 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fceffd700 1 --2- 172.21.15.191:0/3891346906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd015ba40 0x7f6fd00c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.657 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fceffd700 1 -- 172.21.15.191:0/3891346906 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8009b30 con 0x7f6fd00b0760 2022-11-27T16:16:57.657 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fceffd700 1 --2- 172.21.15.191:0/3891346906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 0x7f6fd01f5230 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000e760 tx=0x7f6fc000ea70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:57.658 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fb7fff700 1 -- 172.21.15.191:0/3891346906 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc000c070 con 0x7f6fd00b0760 2022-11-27T16:16:57.658 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fb7fff700 1 -- 172.21.15.191:0/3891346906 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc0013610 con 0x7f6fd00b0760 2022-11-27T16:16:57.658 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fd586b700 1 -- 172.21.15.191:0/3891346906 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd00c8290 con 0x7f6fd00b0760 2022-11-27T16:16:57.658 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fb7fff700 1 -- 172.21.15.191:0/3891346906 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0013980 con 0x7f6fd00b0760 2022-11-27T16:16:57.658 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fd586b700 1 -- 172.21.15.191:0/3891346906 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd00c8850 con 0x7f6fd00b0760 2022-11-27T16:16:57.659 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.638+0000 7f6fb7fff700 1 -- 172.21.15.191:0/3891346906 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f6fc0023070 con 0x7f6fd00b0760 2022-11-27T16:16:57.659 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.639+0000 7f6fb7fff700 1 -- 172.21.15.191:0/3891346906 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f6fc001ccd0 con 0x7f6fd00b0760 2022-11-27T16:16:57.659 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.639+0000 7f6fd586b700 1 -- 172.21.15.191:0/3891346906 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd0073030 con 0x7f6fd00b0760 2022-11-27T16:16:57.659 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.641+0000 7f6fb7fff700 1 -- 172.21.15.191:0/3891346906 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f6fc003b6d0 con 0x7f6fd00b0760 2022-11-27T16:16:57.726 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.723+0000 7f6fd586b700 1 -- 172.21.15.191:0/3891346906 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5bbdd443-3ec5-40fb-86af-3e5a6ab1606a", "id": 2} v 0) v1 -- 0x7f6fd00ce330 con 0x7f6fd00b0760 2022-11-27T16:16:57.733 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.731+0000 7f6fb7fff700 1 -- 172.21.15.191:0/3891346906 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5bbdd443-3ec5-40fb-86af-3e5a6ab1606a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6fc002a030 con 0x7f6fd00b0760 2022-11-27T16:16:57.733 INFO:teuthology.orchestra.run.smithi191.stdout:2 2022-11-27T16:16:57.735 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.732+0000 7f6fd586b700 1 -- 172.21.15.191:0/3891346906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 msgr2=0x7f6fd01f5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.735 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.732+0000 7f6fd586b700 1 --2- 172.21.15.191:0/3891346906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 0x7f6fd01f5230 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000e760 tx=0x7f6fc000ea70 comp rx=0 tx=0).stop 2022-11-27T16:16:57.736 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.732+0000 7f6fd586b700 1 -- 172.21.15.191:0/3891346906 shutdown_connections 2022-11-27T16:16:57.736 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.732+0000 7f6fd586b700 1 --2- 172.21.15.191:0/3891346906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd015ba40 0x7f6fd00c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.736 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.732+0000 7f6fd586b700 1 --2- 172.21.15.191:0/3891346906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fd00b0760 0x7f6fd01f5230 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.736 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.732+0000 7f6fd586b700 1 --2- 172.21.15.191:0/3891346906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fd00cd600 0x7f6fd00b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.737 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.732+0000 7f6fd586b700 1 -- 172.21.15.191:0/3891346906 >> 172.21.15.191:0/3891346906 conn(0x7f6fd00cb050 msgr2=0x7f6fd0159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:57.737 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.732+0000 7f6fd586b700 1 -- 172.21.15.191:0/3891346906 shutdown_connections 2022-11-27T16:16:57.737 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.732+0000 7f6fd586b700 1 -- 172.21.15.191:0/3891346906 wait complete. 2022-11-27T16:16:57.748 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 795fd2fb-0e2e-4419-9b54-ea28ba0ee835 3 2022-11-27T16:16:57.868 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.866+0000 7f47810d7700 1 Processor -- start 2022-11-27T16:16:57.868 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.866+0000 7f47810d7700 1 -- start start 2022-11-27T16:16:57.871 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.867+0000 7f47810d7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c1597f0 0x7f477c0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.872 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.867+0000 7f47810d7700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c071a00 con 0x7f477c0b0760 2022-11-27T16:16:57.872 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.867+0000 7f47810d7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c06ab30 con 0x7f477c15d450 2022-11-27T16:16:57.873 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.867+0000 7f47810d7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0710a0 con 0x7f477c1597f0 2022-11-27T16:16:57.873 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f477ad9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c1597f0 0x7f477c0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.873 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f477a59c700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f477c15d450 0x7f477c0b0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:44676/0 (socket says 172.21.15.191:44676) 2022-11-27T16:16:57.873 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f477a59c700 1 -- 172.21.15.191:0/4056765818 learned_addr learned my addr 172.21.15.191:0/4056765818 (peer_addr_for_me v1:172.21.15.191:0/0) 2022-11-27T16:16:57.874 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f477ad9d700 1 -- 172.21.15.191:0/4056765818 >> v1:172.21.15.191:6789/0 conn(0x7f477c15d450 legacy=0x7f477c0b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.874 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f4779d9b700 1 -- 172.21.15.191:0/4056765818 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154918576 0 0) 0x7f477c06ab30 con 0x7f477c15d450 2022-11-27T16:16:57.874 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f477ad9d700 1 -- 172.21.15.191:0/4056765818 >> v1:172.21.15.134:6789/0 conn(0x7f477c0b0760 legacy=0x7f477c0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.874 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f477ad9d700 1 -- 172.21.15.191:0/4056765818 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c15def0 con 0x7f477c1597f0 2022-11-27T16:16:57.875 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f477ad9d700 1 --2- 172.21.15.191:0/4056765818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c1597f0 0x7f477c0714c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f476c009870 tx=0x7f476c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e77c6a339468dac server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:57.875 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f4779d9b700 1 -- 172.21.15.191:0/4056765818 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c011830 con 0x7f477c1597f0 2022-11-27T16:16:57.875 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f4779d9b700 1 -- 172.21.15.191:0/4056765818 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c011990 con 0x7f477c1597f0 2022-11-27T16:16:57.875 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.868+0000 7f4779d9b700 1 -- 172.21.15.191:0/4056765818 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c011d20 con 0x7f477c1597f0 2022-11-27T16:16:57.875 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.869+0000 7f47810d7700 1 -- 172.21.15.191:0/4056765818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c1597f0 msgr2=0x7f477c0714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.876 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.869+0000 7f47810d7700 1 --2- 172.21.15.191:0/4056765818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c1597f0 0x7f477c0714c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f476c009870 tx=0x7f476c009b80 comp rx=0 tx=0).stop 2022-11-27T16:16:57.876 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.869+0000 7f47810d7700 1 -- 172.21.15.191:0/4056765818 shutdown_connections 2022-11-27T16:16:57.876 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.869+0000 7f47810d7700 1 --2- 172.21.15.191:0/4056765818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c1597f0 0x7f477c0714c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.876 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.869+0000 7f47810d7700 1 -- 172.21.15.191:0/4056765818 >> 172.21.15.191:0/4056765818 conn(0x7f477c0cb050 msgr2=0x7f477c0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:57.877 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.869+0000 7f47810d7700 1 -- 172.21.15.191:0/4056765818 shutdown_connections 2022-11-27T16:16:57.877 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.869+0000 7f47810d7700 1 -- 172.21.15.191:0/4056765818 wait complete. 2022-11-27T16:16:57.880 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.871+0000 7f47810d7700 1 Processor -- start 2022-11-27T16:16:57.881 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.871+0000 7f47810d7700 1 -- start start 2022-11-27T16:16:57.881 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47810d7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b0760 0x7f477c0af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.881 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47810d7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477c1597f0 0x7f477c0af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.882 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47810d7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c15d450 0x7f477c0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:57.882 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47810d7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c15cbd0 con 0x7f477c0b0760 2022-11-27T16:16:57.882 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47810d7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c15c740 con 0x7f477c1597f0 2022-11-27T16:16:57.882 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47810d7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c15c410 con 0x7f477c15d450 2022-11-27T16:16:57.883 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477a59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477c1597f0 0x7f477c0af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.883 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477b59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c15d450 0x7f477c0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.883 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477ad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b0760 0x7f477c0af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:57.883 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477a59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477c1597f0 0x7f477c0af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45456/0 (socket says 172.21.15.191:45456) 2022-11-27T16:16:57.884 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477a59c700 1 -- 172.21.15.191:0/1487068682 learned_addr learned my addr 172.21.15.191:0/1487068682 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:57.884 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477a59c700 1 -- 172.21.15.191:0/1487068682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c15d450 msgr2=0x7f477c0c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.884 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477a59c700 1 --2- 172.21.15.191:0/1487068682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c15d450 0x7f477c0c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.885 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477a59c700 1 -- 172.21.15.191:0/1487068682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b0760 msgr2=0x7f477c0af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.885 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477a59c700 1 --2- 172.21.15.191:0/1487068682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b0760 0x7f477c0af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.885 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477a59c700 1 -- 172.21.15.191:0/1487068682 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0094a0 con 0x7f477c1597f0 2022-11-27T16:16:57.885 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477ad9d700 1 --2- 172.21.15.191:0/1487068682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b0760 0x7f477c0af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:57.886 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477b59e700 1 --2- 172.21.15.191:0/1487068682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c15d450 0x7f477c0c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:57.886 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f477a59c700 1 --2- 172.21.15.191:0/1487068682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477c1597f0 0x7f477c0af1b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f477000c500 tx=0x7f477000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:57.886 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47637fe700 1 -- 172.21.15.191:0/1487068682 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770004c80 con 0x7f477c1597f0 2022-11-27T16:16:57.886 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47637fe700 1 -- 172.21.15.191:0/1487068682 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4770004de0 con 0x7f477c1597f0 2022-11-27T16:16:57.887 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47810d7700 1 -- 172.21.15.191:0/1487068682 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c159d40 con 0x7f477c1597f0 2022-11-27T16:16:57.887 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47637fe700 1 -- 172.21.15.191:0/1487068682 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770007310 con 0x7f477c1597f0 2022-11-27T16:16:57.887 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47810d7700 1 -- 172.21.15.191:0/1487068682 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c15a2e0 con 0x7f477c1597f0 2022-11-27T16:16:57.887 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.872+0000 7f47637fe700 1 -- 172.21.15.191:0/1487068682 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f477001a020 con 0x7f477c1597f0 2022-11-27T16:16:57.888 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.873+0000 7f47637fe700 1 -- 172.21.15.191:0/1487068682 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f4770007470 con 0x7f477c1597f0 2022-11-27T16:16:57.888 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.873+0000 7f47810d7700 1 -- 172.21.15.191:0/1487068682 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c073030 con 0x7f477c1597f0 2022-11-27T16:16:57.888 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.875+0000 7f47637fe700 1 -- 172.21.15.191:0/1487068682 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f47700133c0 con 0x7f477c1597f0 2022-11-27T16:16:57.958 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.956+0000 7f47810d7700 1 -- 172.21.15.191:0/1487068682 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "795fd2fb-0e2e-4419-9b54-ea28ba0ee835", "id": 3} v 0) v1 -- 0x7f477c15c880 con 0x7f477c1597f0 2022-11-27T16:16:57.965 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.963+0000 7f47637fe700 1 -- 172.21.15.191:0/1487068682 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "795fd2fb-0e2e-4419-9b54-ea28ba0ee835", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f477001d7b0 con 0x7f477c1597f0 2022-11-27T16:16:57.966 INFO:teuthology.orchestra.run.smithi191.stdout:3 2022-11-27T16:16:57.967 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.964+0000 7f47810d7700 1 -- 172.21.15.191:0/1487068682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477c1597f0 msgr2=0x7f477c0af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:57.967 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.964+0000 7f47810d7700 1 --2- 172.21.15.191:0/1487068682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477c1597f0 0x7f477c0af1b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f477000c500 tx=0x7f477000c810 comp rx=0 tx=0).stop 2022-11-27T16:16:57.967 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.964+0000 7f47810d7700 1 -- 172.21.15.191:0/1487068682 shutdown_connections 2022-11-27T16:16:57.968 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.964+0000 7f47810d7700 1 --2- 172.21.15.191:0/1487068682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f477c15d450 0x7f477c0c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.968 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.964+0000 7f47810d7700 1 --2- 172.21.15.191:0/1487068682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f477c0b0760 0x7f477c0af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.968 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.964+0000 7f47810d7700 1 --2- 172.21.15.191:0/1487068682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477c1597f0 0x7f477c0af1b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:57.968 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.964+0000 7f47810d7700 1 -- 172.21.15.191:0/1487068682 >> 172.21.15.191:0/1487068682 conn(0x7f477c0cb050 msgr2=0x7f477c072020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:57.969 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.964+0000 7f47810d7700 1 -- 172.21.15.191:0/1487068682 shutdown_connections 2022-11-27T16:16:57.969 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:57.964+0000 7f47810d7700 1 -- 172.21.15.191:0/1487068682 wait complete. 2022-11-27T16:16:57.975 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:57.974+0000 7f75e0047200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T16:16:57.978 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 36005e5e-73a9-4d10-aba3-480165df1594 4 2022-11-27T16:16:58.008 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:58.006+0000 7f46006a3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T16:16:58.100 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.097+0000 7f15a7a65700 1 Processor -- start 2022-11-27T16:16:58.100 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.097+0000 7f15a7a65700 1 -- start start 2022-11-27T16:16:58.108 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a7a65700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f15a00b0760 0x7f15a0069e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.109 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a7a65700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a015b770 con 0x7f15a00ca900 2022-11-27T16:16:58.109 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a7a65700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a00c8dc0 con 0x7f15a015b3a0 2022-11-27T16:16:58.109 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a7a65700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a00cc360 con 0x7f15a00b0760 2022-11-27T16:16:58.109 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a5000700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f15a00b0760 0x7f15a0069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.110 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a5000700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f15a00b0760 0x7f15a0069e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45462/0 (socket says 172.21.15.191:45462) 2022-11-27T16:16:58.110 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a5000700 1 -- 172.21.15.191:0/3761920173 learned_addr learned my addr 172.21.15.191:0/3761920173 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:58.110 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a5000700 1 -- 172.21.15.191:0/3761920173 >> v1:172.21.15.191:6789/0 conn(0x7f15a015b3a0 legacy=0x7f15a00b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.110 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a5000700 1 -- 172.21.15.191:0/3761920173 >> v1:172.21.15.134:6789/0 conn(0x7f15a00ca900 legacy=0x7f15a00cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.111 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a5000700 1 -- 172.21.15.191:0/3761920173 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a00c90a0 con 0x7f15a00b0760 2022-11-27T16:16:58.111 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f15a5000700 1 --2- 172.21.15.191:0/3761920173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 0x7f15a0069e00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f159000bb90 tx=0x7f159000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16f5224893675ba8 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:58.111 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f158ffff700 1 -- 172.21.15.191:0/3761920173 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159000dc40 con 0x7f15a00b0760 2022-11-27T16:16:58.112 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.099+0000 7f158ffff700 1 -- 172.21.15.191:0/3761920173 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159000dda0 con 0x7f15a00b0760 2022-11-27T16:16:58.112 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.100+0000 7f158ffff700 1 -- 172.21.15.191:0/3761920173 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590017630 con 0x7f15a00b0760 2022-11-27T16:16:58.113 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.100+0000 7f15a7a65700 1 -- 172.21.15.191:0/3761920173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 msgr2=0x7f15a0069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.113 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.100+0000 7f15a7a65700 1 --2- 172.21.15.191:0/3761920173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 0x7f15a0069e00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f159000bb90 tx=0x7f159000bea0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.114 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.100+0000 7f15a7a65700 1 -- 172.21.15.191:0/3761920173 shutdown_connections 2022-11-27T16:16:58.114 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.100+0000 7f15a7a65700 1 --2- 172.21.15.191:0/3761920173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 0x7f15a0069e00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.115 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.100+0000 7f15a7a65700 1 -- 172.21.15.191:0/3761920173 >> 172.21.15.191:0/3761920173 conn(0x7f15a00cb050 msgr2=0x7f15a00cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:58.115 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.101+0000 7f15a7a65700 1 -- 172.21.15.191:0/3761920173 shutdown_connections 2022-11-27T16:16:58.116 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.101+0000 7f15a7a65700 1 -- 172.21.15.191:0/3761920173 wait complete. 2022-11-27T16:16:58.116 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.102+0000 7f15a7a65700 1 Processor -- start 2022-11-27T16:16:58.116 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.102+0000 7f15a7a65700 1 -- start start 2022-11-27T16:16:58.117 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.103+0000 7f15a7a65700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 0x7f15a014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.117 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.103+0000 7f15a7a65700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a00ca900 0x7f15a0157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.118 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.103+0000 7f15a7a65700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15a015b3a0 0x7f15a00b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.118 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.103+0000 7f15a7a65700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00c5170 con 0x7f15a00ca900 2022-11-27T16:16:58.118 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.103+0000 7f15a7a65700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00b2b60 con 0x7f15a00b0760 2022-11-27T16:16:58.119 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.103+0000 7f15a7a65700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00b2e00 con 0x7f15a015b3a0 2022-11-27T16:16:58.119 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5801700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 0x7f15a014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.120 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a6002700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15a015b3a0 0x7f15a00b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.120 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5801700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 0x7f15a014e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45466/0 (socket says 172.21.15.191:45466) 2022-11-27T16:16:58.120 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5801700 1 -- 172.21.15.191:0/171802128 learned_addr learned my addr 172.21.15.191:0/171802128 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:58.121 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5000700 1 --2- 172.21.15.191:0/171802128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a00ca900 0x7f15a0157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.121 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5801700 1 -- 172.21.15.191:0/171802128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15a015b3a0 msgr2=0x7f15a00b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.122 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5801700 1 --2- 172.21.15.191:0/171802128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15a015b3a0 0x7f15a00b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.122 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5801700 1 -- 172.21.15.191:0/171802128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a00ca900 msgr2=0x7f15a0157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.123 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5801700 1 --2- 172.21.15.191:0/171802128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a00ca900 0x7f15a0157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.123 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5801700 1 -- 172.21.15.191:0/171802128 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159000b7f0 con 0x7f15a00b0760 2022-11-27T16:16:58.123 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a6002700 1 --2- 172.21.15.191:0/171802128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15a015b3a0 0x7f15a00b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:58.123 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5000700 1 --2- 172.21.15.191:0/171802128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a00ca900 0x7f15a0157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:58.124 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a5801700 1 --2- 172.21.15.191:0/171802128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 0x7f15a014e9e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c450 tx=0x7f159c00c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:58.124 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f158dffb700 1 -- 172.21.15.191:0/171802128 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c0052d0 con 0x7f15a00b0760 2022-11-27T16:16:58.124 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f158dffb700 1 -- 172.21.15.191:0/171802128 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c005430 con 0x7f15a00b0760 2022-11-27T16:16:58.124 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f15a7a65700 1 -- 172.21.15.191:0/171802128 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a00b30e0 con 0x7f15a00b0760 2022-11-27T16:16:58.125 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.104+0000 7f158dffb700 1 -- 172.21.15.191:0/171802128 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c0038c0 con 0x7f15a00b0760 2022-11-27T16:16:58.125 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.105+0000 7f158dffb700 1 -- 172.21.15.191:0/171802128 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f159c019030 con 0x7f15a00b0760 2022-11-27T16:16:58.126 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.105+0000 7f15a7a65700 1 -- 172.21.15.191:0/171802128 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a014ef20 con 0x7f15a00b0760 2022-11-27T16:16:58.126 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.105+0000 7f158dffb700 1 -- 172.21.15.191:0/171802128 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f159c0117c0 con 0x7f15a00b0760 2022-11-27T16:16:58.126 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.105+0000 7f15a7a65700 1 -- 172.21.15.191:0/171802128 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a00b3600 con 0x7f15a00b0760 2022-11-27T16:16:58.127 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.107+0000 7f158dffb700 1 -- 172.21.15.191:0/171802128 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f159c017070 con 0x7f15a00b0760 2022-11-27T16:16:58.191 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.189+0000 7f15a7a65700 1 -- 172.21.15.191:0/171802128 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "36005e5e-73a9-4d10-aba3-480165df1594", "id": 4} v 0) v1 -- 0x7f15a0073030 con 0x7f15a00b0760 2022-11-27T16:16:58.199 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.197+0000 7f158dffb700 1 -- 172.21.15.191:0/171802128 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "36005e5e-73a9-4d10-aba3-480165df1594", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f159c003a20 con 0x7f15a00b0760 2022-11-27T16:16:58.199 INFO:teuthology.orchestra.run.smithi191.stdout:4 2022-11-27T16:16:58.200 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.198+0000 7f15a7a65700 1 -- 172.21.15.191:0/171802128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 msgr2=0x7f15a014e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.201 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.198+0000 7f15a7a65700 1 --2- 172.21.15.191:0/171802128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 0x7f15a014e9e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c450 tx=0x7f159c00c760 comp rx=0 tx=0).stop 2022-11-27T16:16:58.201 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.198+0000 7f15a7a65700 1 -- 172.21.15.191:0/171802128 shutdown_connections 2022-11-27T16:16:58.201 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.198+0000 7f15a7a65700 1 --2- 172.21.15.191:0/171802128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15a015b3a0 0x7f15a00b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.201 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.198+0000 7f15a7a65700 1 --2- 172.21.15.191:0/171802128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a00b0760 0x7f15a014e9e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.202 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.198+0000 7f15a7a65700 1 --2- 172.21.15.191:0/171802128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a00ca900 0x7f15a0157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.202 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.198+0000 7f15a7a65700 1 -- 172.21.15.191:0/171802128 >> 172.21.15.191:0/171802128 conn(0x7f15a00cb050 msgr2=0x7f15a00ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:58.202 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.198+0000 7f15a7a65700 1 -- 172.21.15.191:0/171802128 shutdown_connections 2022-11-27T16:16:58.202 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.198+0000 7f15a7a65700 1 -- 172.21.15.191:0/171802128 wait complete. 2022-11-27T16:16:58.212 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 6f5b5e55-e69b-4097-a47c-cd7fc0681dbd 5 2022-11-27T16:16:58.374 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.373+0000 7f82fe3fc700 1 Processor -- start 2022-11-27T16:16:58.375 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.373+0000 7f82fe3fc700 1 -- start start 2022-11-27T16:16:58.381 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82fe3fc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f82f81598c0 0x7f82f80d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.382 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82fe3fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80b34d0 0x7f82f806d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.382 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82fe3fc700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f80b1ca0 con 0x7f82f80d0ae0 2022-11-27T16:16:58.382 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82fe3fc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f80cc830 con 0x7f82f81598c0 2022-11-27T16:16:58.382 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82fe3fc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80cc270 con 0x7f82f80b34d0 2022-11-27T16:16:58.383 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82f7fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f82f81598c0 0x7f82f80d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.383 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82fc999700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80b34d0 0x7f82f806d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.383 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82f7fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f82f81598c0 0x7f82f80d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45472/0 (socket says 172.21.15.191:45472) 2022-11-27T16:16:58.383 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82f7fff700 1 -- 172.21.15.191:0/3926948343 learned_addr learned my addr 172.21.15.191:0/3926948343 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:58.384 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82f7fff700 1 -- 172.21.15.191:0/3926948343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80b34d0 msgr2=0x7f82f806d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:16:58.384 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82f7fff700 1 --2- 172.21.15.191:0/3926948343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80b34d0 0x7f82f806d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.384 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82f7fff700 1 -- 172.21.15.191:0/3926948343 >> v1:172.21.15.134:6789/0 conn(0x7f82f80d0ae0 legacy=0x7f82f80b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.384 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82f7fff700 1 -- 172.21.15.191:0/3926948343 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80cc550 con 0x7f82f81598c0 2022-11-27T16:16:58.384 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82fc999700 1 --2- 172.21.15.191:0/3926948343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80b34d0 0x7f82f806d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:58.385 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82f7fff700 1 --2- 172.21.15.191:0/3926948343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f81598c0 0x7f82f80d05a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098f0 tx=0x7f82e8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506998a747a53000 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:58.385 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.374+0000 7f82f6ffd700 1 -- 172.21.15.191:0/3926948343 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8011830 con 0x7f82f81598c0 2022-11-27T16:16:58.385 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.375+0000 7f82f6ffd700 1 -- 172.21.15.191:0/3926948343 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8011990 con 0x7f82f81598c0 2022-11-27T16:16:58.385 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.375+0000 7f82f6ffd700 1 -- 172.21.15.191:0/3926948343 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8011d20 con 0x7f82f81598c0 2022-11-27T16:16:58.386 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.375+0000 7f82fe3fc700 1 -- 172.21.15.191:0/3926948343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f81598c0 msgr2=0x7f82f80d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.386 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.375+0000 7f82fe3fc700 1 --2- 172.21.15.191:0/3926948343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f81598c0 0x7f82f80d05a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098f0 tx=0x7f82e8009c00 comp rx=0 tx=0).stop 2022-11-27T16:16:58.386 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.375+0000 7f82fe3fc700 1 -- 172.21.15.191:0/3926948343 shutdown_connections 2022-11-27T16:16:58.386 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.375+0000 7f82fe3fc700 1 --2- 172.21.15.191:0/3926948343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80b34d0 0x7f82f806d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.386 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.375+0000 7f82fe3fc700 1 --2- 172.21.15.191:0/3926948343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f81598c0 0x7f82f80d05a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.387 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.375+0000 7f82fe3fc700 1 -- 172.21.15.191:0/3926948343 >> 172.21.15.191:0/3926948343 conn(0x7f82f8071060 msgr2=0x7f82f8071430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:58.387 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.375+0000 7f82fe3fc700 1 -- 172.21.15.191:0/3926948343 shutdown_connections 2022-11-27T16:16:58.387 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.375+0000 7f82fe3fc700 1 -- 172.21.15.191:0/3926948343 wait complete. 2022-11-27T16:16:58.387 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.376+0000 7f82fe3fc700 1 Processor -- start 2022-11-27T16:16:58.388 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82fe3fc700 1 -- start start 2022-11-27T16:16:58.388 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82fe3fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f80b34d0 0x7f82f814cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.388 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82fe3fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80d0ae0 0x7f82f8150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.389 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82fe3fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f81598c0 0x7f82f815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.389 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82fe3fc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80c70f0 con 0x7f82f81598c0 2022-11-27T16:16:58.389 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82fe3fc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f814d4d0 con 0x7f82f80b34d0 2022-11-27T16:16:58.389 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82fe3fc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f8153bf0 con 0x7f82f80d0ae0 2022-11-27T16:16:58.390 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82f7fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f80b34d0 0x7f82f814cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.390 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82f7fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f80b34d0 0x7f82f814cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45478/0 (socket says 172.21.15.191:45478) 2022-11-27T16:16:58.390 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82f7fff700 1 -- 172.21.15.191:0/1366078208 learned_addr learned my addr 172.21.15.191:0/1366078208 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:58.391 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.377+0000 7f82f77fe700 1 --2- 172.21.15.191:0/1366078208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80d0ae0 0x7f82f8150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.391 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82fc999700 1 --2- 172.21.15.191:0/1366078208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f81598c0 0x7f82f815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.392 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f7fff700 1 -- 172.21.15.191:0/1366078208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80d0ae0 msgr2=0x7f82f8150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.392 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f7fff700 1 --2- 172.21.15.191:0/1366078208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80d0ae0 0x7f82f8150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.392 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f7fff700 1 -- 172.21.15.191:0/1366078208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f81598c0 msgr2=0x7f82f815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.393 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f7fff700 1 --2- 172.21.15.191:0/1366078208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f81598c0 0x7f82f815a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.393 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f7fff700 1 -- 172.21.15.191:0/1366078208 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f0009a70 con 0x7f82f80b34d0 2022-11-27T16:16:58.393 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f7fff700 1 --2- 172.21.15.191:0/1366078208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f80b34d0 0x7f82f814cf90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009c40 tx=0x7f82e8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:58.394 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f4ff9700 1 -- 172.21.15.191:0/1366078208 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e80044d0 con 0x7f82f80b34d0 2022-11-27T16:16:58.394 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f4ff9700 1 -- 172.21.15.191:0/1366078208 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e801e070 con 0x7f82f80b34d0 2022-11-27T16:16:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82fe3fc700 1 -- 172.21.15.191:0/1366078208 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e8009550 con 0x7f82f80b34d0 2022-11-27T16:16:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f4ff9700 1 -- 172.21.15.191:0/1366078208 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e80038e0 con 0x7f82f80b34d0 2022-11-27T16:16:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82fe3fc700 1 -- 172.21.15.191:0/1366078208 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f81541b0 con 0x7f82f80b34d0 2022-11-27T16:16:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f4ff9700 1 -- 172.21.15.191:0/1366078208 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f82e800f050 con 0x7f82f80b34d0 2022-11-27T16:16:58.396 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.378+0000 7f82f4ff9700 1 -- 172.21.15.191:0/1366078208 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f82e802dbf0 con 0x7f82f80b34d0 2022-11-27T16:16:58.396 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.379+0000 7f82fe3fc700 1 -- 172.21.15.191:0/1366078208 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f8154340 con 0x7f82f80b34d0 2022-11-27T16:16:58.397 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.381+0000 7f82f4ff9700 1 -- 172.21.15.191:0/1366078208 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f82e8020070 con 0x7f82f80b34d0 2022-11-27T16:16:58.498 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.497+0000 7f82fe3fc700 1 -- 172.21.15.191:0/1366078208 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6f5b5e55-e69b-4097-a47c-cd7fc0681dbd", "id": 5} v 0) v1 -- 0x7f82f8073030 con 0x7f82f80b34d0 2022-11-27T16:16:58.504 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.502+0000 7f82f4ff9700 1 -- 172.21.15.191:0/1366078208 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6f5b5e55-e69b-4097-a47c-cd7fc0681dbd", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f82e8015020 con 0x7f82f80b34d0 2022-11-27T16:16:58.505 INFO:teuthology.orchestra.run.smithi191.stdout:5 2022-11-27T16:16:58.507 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.503+0000 7f82fe3fc700 1 -- 172.21.15.191:0/1366078208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f80b34d0 msgr2=0x7f82f814cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.507 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.503+0000 7f82fe3fc700 1 --2- 172.21.15.191:0/1366078208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f80b34d0 0x7f82f814cf90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009c40 tx=0x7f82e8004750 comp rx=0 tx=0).stop 2022-11-27T16:16:58.507 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.503+0000 7f82fe3fc700 1 -- 172.21.15.191:0/1366078208 shutdown_connections 2022-11-27T16:16:58.508 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.503+0000 7f82fe3fc700 1 --2- 172.21.15.191:0/1366078208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f81598c0 0x7f82f815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.508 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.504+0000 7f82fe3fc700 1 --2- 172.21.15.191:0/1366078208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82f80b34d0 0x7f82f814cf90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.509 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.504+0000 7f82fe3fc700 1 --2- 172.21.15.191:0/1366078208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82f80d0ae0 0x7f82f8150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.509 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.504+0000 7f82fe3fc700 1 -- 172.21.15.191:0/1366078208 >> 172.21.15.191:0/1366078208 conn(0x7f82f8071060 msgr2=0x7f82f80cba60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:58.510 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.504+0000 7f82fe3fc700 1 -- 172.21.15.191:0/1366078208 shutdown_connections 2022-11-27T16:16:58.510 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.504+0000 7f82fe3fc700 1 -- 172.21.15.191:0/1366078208 wait complete. 2022-11-27T16:16:58.517 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 074077c9-fc46-4f79-8368-5a96eba6da06 6 2022-11-27T16:16:58.527 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:58.525+0000 7f75e0047200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T16:16:58.584 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:58.582+0000 7f46006a3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T16:16:58.640 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.638+0000 7fc5e3099700 1 Processor -- start 2022-11-27T16:16:58.640 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.638+0000 7fc5e3099700 1 -- start start 2022-11-27T16:16:58.644 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5e3099700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.644 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5e3099700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5dc06d920 0x7fc5dc15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.644 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5e3099700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5dc0693f0 con 0x7fc5dc0ce7b0 2022-11-27T16:16:58.645 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5e3099700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5dc06dda0 con 0x7fc5dc06d920 2022-11-27T16:16:58.645 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5e3099700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc06a1c0 con 0x7fc5dc0cf530 2022-11-27T16:16:58.645 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5e1636700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5dc06d920 0x7fc5dc15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.646 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5e1636700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5dc06d920 0x7fc5dc15d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45488/0 (socket says 172.21.15.191:45488) 2022-11-27T16:16:58.646 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5dbfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.646 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5e1636700 1 -- 172.21.15.191:0/1834795613 learned_addr learned my addr 172.21.15.191:0/1834795613 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:58.646 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5dbfff700 1 -- 172.21.15.191:0/1834795613 >> v2:172.21.15.191:3300/0 conn(0x7fc5dc06d920 msgr2=0x7fc5dc15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:16:58.646 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5dbfff700 1 --2- 172.21.15.191:0/1834795613 >> v2:172.21.15.191:3300/0 conn(0x7fc5dc06d920 0x7fc5dc15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.647 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5dbfff700 1 -- 172.21.15.191:0/1834795613 >> v1:172.21.15.134:6789/0 conn(0x7fc5dc0ce7b0 legacy=0x7fc5dc159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.647 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5dbfff700 1 -- 172.21.15.191:0/1834795613 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc069640 con 0x7fc5dc0cf530 2022-11-27T16:16:58.647 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.639+0000 7fc5e1636700 1 --2- 172.21.15.191:0/1834795613 >> v2:172.21.15.191:3300/0 conn(0x7fc5dc06d920 0x7fc5dc15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:58.647 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.640+0000 7fc5dbfff700 1 --2- 172.21.15.191:0/1834795613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc06d3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009840 tx=0x7fc5c8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6aa99f1da1ac2a3 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:58.648 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.640+0000 7fc5db7fe700 1 -- 172.21.15.191:0/1834795613 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8011830 con 0x7fc5dc0cf530 2022-11-27T16:16:58.648 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.640+0000 7fc5db7fe700 1 -- 172.21.15.191:0/1834795613 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8011990 con 0x7fc5dc0cf530 2022-11-27T16:16:58.648 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.640+0000 7fc5db7fe700 1 -- 172.21.15.191:0/1834795613 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8011d20 con 0x7fc5dc0cf530 2022-11-27T16:16:58.648 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.640+0000 7fc5e3099700 1 -- 172.21.15.191:0/1834795613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 msgr2=0x7fc5dc06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.648 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.640+0000 7fc5e3099700 1 --2- 172.21.15.191:0/1834795613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc06d3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009840 tx=0x7fc5c8009b50 comp rx=0 tx=0).stop 2022-11-27T16:16:58.649 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.640+0000 7fc5e3099700 1 -- 172.21.15.191:0/1834795613 shutdown_connections 2022-11-27T16:16:58.649 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.640+0000 7fc5e3099700 1 --2- 172.21.15.191:0/1834795613 >> v2:172.21.15.191:3300/0 conn(0x7fc5dc06d920 0x7fc5dc15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.649 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.640+0000 7fc5e3099700 1 --2- 172.21.15.191:0/1834795613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc06d3e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.649 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.640+0000 7fc5e3099700 1 -- 172.21.15.191:0/1834795613 >> 172.21.15.191:0/1834795613 conn(0x7fc5dc0b16c0 msgr2=0x7fc5dc0cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:58.650 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.641+0000 7fc5e3099700 1 -- 172.21.15.191:0/1834795613 shutdown_connections 2022-11-27T16:16:58.650 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.641+0000 7fc5e3099700 1 -- 172.21.15.191:0/1834795613 wait complete. 2022-11-27T16:16:58.650 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.642+0000 7fc5e3099700 1 Processor -- start 2022-11-27T16:16:58.650 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.642+0000 7fc5e3099700 1 -- start start 2022-11-27T16:16:58.653 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e3099700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5dc06d920 0x7fc5dc0c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.654 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e3099700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5dc0ce7b0 0x7fc5dc1476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.654 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e3099700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc1e3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.654 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e3099700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc0c7c60 con 0x7fc5dc0ce7b0 2022-11-27T16:16:58.654 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e3099700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc0c6fe0 con 0x7fc5dc06d920 2022-11-27T16:16:58.655 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e3099700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc0c72c0 con 0x7fc5dc0cf530 2022-11-27T16:16:58.655 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e1636700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc1e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.655 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e0e35700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5dc06d920 0x7fc5dc0c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.655 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e1636700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc1e3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:54256/0 (socket says 172.21.15.191:54256) 2022-11-27T16:16:58.656 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e0e35700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5dc06d920 0x7fc5dc0c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45490/0 (socket says 172.21.15.191:45490) 2022-11-27T16:16:58.656 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5e1636700 1 -- 172.21.15.191:0/1121954121 learned_addr learned my addr 172.21.15.191:0/1121954121 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:58.656 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.643+0000 7fc5dbfff700 1 --2- 172.21.15.191:0/1121954121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5dc0ce7b0 0x7fc5dc1476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.656 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5e1636700 1 -- 172.21.15.191:0/1121954121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5dc06d920 msgr2=0x7fc5dc0c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.656 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5e1636700 1 --2- 172.21.15.191:0/1121954121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5dc06d920 0x7fc5dc0c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.657 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5e1636700 1 -- 172.21.15.191:0/1121954121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5dc0ce7b0 msgr2=0x7fc5dc1476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.657 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5e1636700 1 --2- 172.21.15.191:0/1121954121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5dc0ce7b0 0x7fc5dc1476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.657 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5e1636700 1 -- 172.21.15.191:0/1121954121 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d4009b30 con 0x7fc5dc0cf530 2022-11-27T16:16:58.657 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5e0e35700 1 --2- 172.21.15.191:0/1121954121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5dc06d920 0x7fc5dc0c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:58.658 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5e1636700 1 --2- 172.21.15.191:0/1121954121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc1e3760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400efc0 tx=0x7fc5d400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:58.658 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5d97fa700 1 -- 172.21.15.191:0/1121954121 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4018070 con 0x7fc5dc0cf530 2022-11-27T16:16:58.658 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5d97fa700 1 -- 172.21.15.191:0/1121954121 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d400f040 con 0x7fc5dc0cf530 2022-11-27T16:16:58.658 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5d97fa700 1 -- 172.21.15.191:0/1121954121 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4003c20 con 0x7fc5dc0cf530 2022-11-27T16:16:58.658 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5e3099700 1 -- 172.21.15.191:0/1121954121 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c80094a0 con 0x7fc5dc0cf530 2022-11-27T16:16:58.659 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5e3099700 1 -- 172.21.15.191:0/1121954121 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5dc0c7880 con 0x7fc5dc0cf530 2022-11-27T16:16:58.659 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5d97fa700 1 -- 172.21.15.191:0/1121954121 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc5d400c880 con 0x7fc5dc0cf530 2022-11-27T16:16:58.659 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.644+0000 7fc5d97fa700 1 -- 172.21.15.191:0/1121954121 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fc5d40092e0 con 0x7fc5dc0cf530 2022-11-27T16:16:58.659 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.645+0000 7fc5e3099700 1 -- 172.21.15.191:0/1121954121 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5dc0c7da0 con 0x7fc5dc0cf530 2022-11-27T16:16:58.660 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.647+0000 7fc5d97fa700 1 -- 172.21.15.191:0/1121954121 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fc5d4003e30 con 0x7fc5dc0cf530 2022-11-27T16:16:58.683 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:58.681+0000 7f75e0047200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T16:16:58.732 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.731+0000 7fc5e3099700 1 -- 172.21.15.191:0/1121954121 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "074077c9-fc46-4f79-8368-5a96eba6da06", "id": 6} v 0) v1 -- 0x7fc5dc065ac0 con 0x7fc5dc0cf530 2022-11-27T16:16:58.738 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.737+0000 7fc5d97fa700 1 -- 172.21.15.191:0/1121954121 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "074077c9-fc46-4f79-8368-5a96eba6da06", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc5d402c0b0 con 0x7fc5dc0cf530 2022-11-27T16:16:58.739 INFO:teuthology.orchestra.run.smithi191.stdout:6 2022-11-27T16:16:58.740 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.738+0000 7fc5e3099700 1 -- 172.21.15.191:0/1121954121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 msgr2=0x7fc5dc1e3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.740 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.738+0000 7fc5e3099700 1 --2- 172.21.15.191:0/1121954121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc1e3760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400efc0 tx=0x7fc5d400c3a0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.741 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.738+0000 7fc5e3099700 1 -- 172.21.15.191:0/1121954121 shutdown_connections 2022-11-27T16:16:58.741 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.738+0000 7fc5e3099700 1 --2- 172.21.15.191:0/1121954121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5dc0cf530 0x7fc5dc1e3760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.741 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.738+0000 7fc5e3099700 1 --2- 172.21.15.191:0/1121954121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5dc06d920 0x7fc5dc0c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.741 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.738+0000 7fc5e3099700 1 --2- 172.21.15.191:0/1121954121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5dc0ce7b0 0x7fc5dc1476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.742 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.738+0000 7fc5e3099700 1 -- 172.21.15.191:0/1121954121 >> 172.21.15.191:0/1121954121 conn(0x7fc5dc0b16c0 msgr2=0x7fc5dc15a830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:58.742 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.738+0000 7fc5e3099700 1 -- 172.21.15.191:0/1121954121 shutdown_connections 2022-11-27T16:16:58.742 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.738+0000 7fc5e3099700 1 -- 172.21.15.191:0/1121954121 wait complete. 2022-11-27T16:16:58.746 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:58.744+0000 7f46006a3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T16:16:58.751 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 112889bf-b637-4500-a30d-03e23e8a3267 7 2022-11-27T16:16:58.872 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.870+0000 7f8b7c103700 1 Processor -- start 2022-11-27T16:16:58.873 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.870+0000 7f8b7c103700 1 -- start start 2022-11-27T16:16:58.879 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.871+0000 7f8b7c103700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8b741598c0 0x7f8b740d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.880 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.871+0000 7f8b7c103700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b740b1ca0 con 0x7f8b740b34d0 2022-11-27T16:16:58.881 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.871+0000 7f8b7c103700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b740cc270 con 0x7f8b740d0ae0 2022-11-27T16:16:58.881 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.871+0000 7f8b7c103700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b740cc830 con 0x7f8b741598c0 2022-11-27T16:16:58.881 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b7969e700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f8b740d0ae0 0x7f8b740b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:44724/0 (socket says 172.21.15.191:44724) 2022-11-27T16:16:58.881 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b7969e700 1 -- 172.21.15.191:0/2434548586 learned_addr learned my addr 172.21.15.191:0/2434548586 (peer_addr_for_me v1:172.21.15.191:0/0) 2022-11-27T16:16:58.882 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b79e9f700 1 --2- 172.21.15.191:0/2434548586 >> v2:172.21.15.134:3300/0 conn(0x7f8b741598c0 0x7f8b740d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.882 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b78e9d700 1 -- 172.21.15.191:0/2434548586 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516246229 0 0) 0x7f8b740cc270 con 0x7f8b740d0ae0 2022-11-27T16:16:58.882 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b78e9d700 1 -- 172.21.15.191:0/2434548586 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b5c0035b0 con 0x7f8b740d0ae0 2022-11-27T16:16:58.882 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b78e9d700 1 -- 172.21.15.191:0/2434548586 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927315847 0 0) 0x7f8b740b1ca0 con 0x7f8b740b34d0 2022-11-27T16:16:58.882 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b78e9d700 1 -- 172.21.15.191:0/2434548586 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b740cc270 con 0x7f8b740b34d0 2022-11-27T16:16:58.883 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b79e9f700 1 -- 172.21.15.191:0/2434548586 >> v1:172.21.15.191:6789/0 conn(0x7f8b740d0ae0 legacy=0x7f8b740b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.883 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b79e9f700 1 -- 172.21.15.191:0/2434548586 >> v1:172.21.15.134:6789/0 conn(0x7f8b740b34d0 legacy=0x7f8b7406d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.883 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b79e9f700 1 -- 172.21.15.191:0/2434548586 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740cc550 con 0x7f8b741598c0 2022-11-27T16:16:58.883 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b79e9f700 1 --2- 172.21.15.191:0/2434548586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b741598c0 0x7f8b740d05a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009840 tx=0x7f8b70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6d21d18fdba21c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:58.884 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b78e9d700 1 -- 172.21.15.191:0/2434548586 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70011830 con 0x7f8b741598c0 2022-11-27T16:16:58.884 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b78e9d700 1 -- 172.21.15.191:0/2434548586 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b70011990 con 0x7f8b741598c0 2022-11-27T16:16:58.884 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.872+0000 7f8b78e9d700 1 -- 172.21.15.191:0/2434548586 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70011d20 con 0x7f8b741598c0 2022-11-27T16:16:58.885 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.873+0000 7f8b7c103700 1 -- 172.21.15.191:0/2434548586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b741598c0 msgr2=0x7f8b740d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.885 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.873+0000 7f8b7c103700 1 --2- 172.21.15.191:0/2434548586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b741598c0 0x7f8b740d05a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009840 tx=0x7f8b70009b50 comp rx=0 tx=0).stop 2022-11-27T16:16:58.885 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.873+0000 7f8b7c103700 1 -- 172.21.15.191:0/2434548586 shutdown_connections 2022-11-27T16:16:58.885 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.873+0000 7f8b7c103700 1 --2- 172.21.15.191:0/2434548586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b741598c0 0x7f8b740d05a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.886 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.873+0000 7f8b7c103700 1 -- 172.21.15.191:0/2434548586 >> 172.21.15.191:0/2434548586 conn(0x7f8b74071060 msgr2=0x7f8b74071430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:58.886 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.873+0000 7f8b7c103700 1 -- 172.21.15.191:0/2434548586 shutdown_connections 2022-11-27T16:16:58.886 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.873+0000 7f8b7c103700 1 -- 172.21.15.191:0/2434548586 wait complete. 2022-11-27T16:16:58.887 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.874+0000 7f8b7c103700 1 Processor -- start 2022-11-27T16:16:58.887 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.874+0000 7f8b7c103700 1 -- start start 2022-11-27T16:16:58.887 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7c103700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b740b34d0 0x7f8b74156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.887 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7c103700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b740d0ae0 0x7f8b7415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.888 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7c103700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b741598c0 0x7f8b74155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:58.888 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7c103700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7415c480 con 0x7f8b741598c0 2022-11-27T16:16:58.888 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7c103700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7415b9f0 con 0x7f8b740d0ae0 2022-11-27T16:16:58.889 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7c103700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740c7cf0 con 0x7f8b740b34d0 2022-11-27T16:16:58.889 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7969e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b740d0ae0 0x7f8b7415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.889 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b79e9f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b740b34d0 0x7f8b74156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:58.889 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7969e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b740d0ae0 0x7f8b7415bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:45502/0 (socket says 172.21.15.191:45502) 2022-11-27T16:16:58.890 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7969e700 1 -- 172.21.15.191:0/107764071 learned_addr learned my addr 172.21.15.191:0/107764071 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-11-27T16:16:58.890 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7969e700 1 -- 172.21.15.191:0/107764071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b740b34d0 msgr2=0x7f8b74156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.890 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7969e700 1 --2- 172.21.15.191:0/107764071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b740b34d0 0x7f8b74156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.890 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7969e700 1 -- 172.21.15.191:0/107764071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b741598c0 msgr2=0x7f8b74155670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2022-11-27T16:16:58.890 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7969e700 1 --2- 172.21.15.191:0/107764071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b741598c0 0x7f8b74155670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.891 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7969e700 1 -- 172.21.15.191:0/107764071 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700094a0 con 0x7f8b740d0ae0 2022-11-27T16:16:58.891 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.875+0000 7f8b7969e700 1 --2- 172.21.15.191:0/107764071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b740d0ae0 0x7f8b7415bf40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c700 tx=0x7f8b6400ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:58.891 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.876+0000 7f8b627fc700 1 -- 172.21.15.191:0/107764071 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64005d70 con 0x7f8b740d0ae0 2022-11-27T16:16:58.891 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.876+0000 7f8b627fc700 1 -- 172.21.15.191:0/107764071 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b640070d0 con 0x7f8b740d0ae0 2022-11-27T16:16:58.892 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.876+0000 7f8b7c103700 1 -- 172.21.15.191:0/107764071 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b74157200 con 0x7f8b740d0ae0 2022-11-27T16:16:58.892 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.876+0000 7f8b627fc700 1 -- 172.21.15.191:0/107764071 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64007460 con 0x7f8b740d0ae0 2022-11-27T16:16:58.892 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.876+0000 7f8b7c103700 1 -- 172.21.15.191:0/107764071 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b741577c0 con 0x7f8b740d0ae0 2022-11-27T16:16:58.892 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.876+0000 7f8b627fc700 1 -- 172.21.15.191:0/107764071 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8b64019030 con 0x7f8b740d0ae0 2022-11-27T16:16:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.876+0000 7f8b627fc700 1 -- 172.21.15.191:0/107764071 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f8b6401c440 con 0x7f8b740d0ae0 2022-11-27T16:16:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.876+0000 7f8b7c103700 1 -- 172.21.15.191:0/107764071 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b740c7ac0 con 0x7f8b740d0ae0 2022-11-27T16:16:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.878+0000 7f8b627fc700 1 -- 172.21.15.191:0/107764071 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f8b740c7ac0 con 0x7f8b740d0ae0 2022-11-27T16:16:58.963 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.962+0000 7f8b7c103700 1 -- 172.21.15.191:0/107764071 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "112889bf-b637-4500-a30d-03e23e8a3267", "id": 7} v 0) v1 -- 0x7f8b74073030 con 0x7f8b740d0ae0 2022-11-27T16:16:58.976 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.974+0000 7f8b627fc700 1 -- 172.21.15.191:0/107764071 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "112889bf-b637-4500-a30d-03e23e8a3267", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8b64021060 con 0x7f8b740d0ae0 2022-11-27T16:16:58.976 INFO:teuthology.orchestra.run.smithi191.stdout:7 2022-11-27T16:16:58.977 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.975+0000 7f8b7c103700 1 -- 172.21.15.191:0/107764071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b740d0ae0 msgr2=0x7f8b7415bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:58.978 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.975+0000 7f8b7c103700 1 --2- 172.21.15.191:0/107764071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b740d0ae0 0x7f8b7415bf40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c700 tx=0x7f8b6400ca10 comp rx=0 tx=0).stop 2022-11-27T16:16:58.978 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.975+0000 7f8b7c103700 1 -- 172.21.15.191:0/107764071 shutdown_connections 2022-11-27T16:16:58.978 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.975+0000 7f8b7c103700 1 --2- 172.21.15.191:0/107764071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b741598c0 0x7f8b74155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.978 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.975+0000 7f8b7c103700 1 --2- 172.21.15.191:0/107764071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b740b34d0 0x7f8b74156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.979 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.975+0000 7f8b7c103700 1 --2- 172.21.15.191:0/107764071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b740d0ae0 0x7f8b7415bf40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:58.979 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.975+0000 7f8b7c103700 1 -- 172.21.15.191:0/107764071 >> 172.21.15.191:0/107764071 conn(0x7f8b74071060 msgr2=0x7f8b7406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:58.979 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.975+0000 7f8b7c103700 1 -- 172.21.15.191:0/107764071 shutdown_connections 2022-11-27T16:16:58.979 INFO:teuthology.orchestra.run.smithi191.stderr:2022-11-27T16:16:58.975+0000 7f8b7c103700 1 -- 172.21.15.191:0/107764071 wait complete. 2022-11-27T16:16:58.989 INFO:tasks.ceph.osd.0:Restarting daemon 2022-11-27T16:16:58.989 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-11-27T16:16:58.992 INFO:tasks.ceph.osd.0:Started 2022-11-27T16:16:58.992 INFO:tasks.ceph.osd.1:Restarting daemon 2022-11-27T16:16:58.992 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-11-27T16:16:58.995 INFO:tasks.ceph.osd.1:Started 2022-11-27T16:16:58.995 INFO:tasks.ceph.osd.2:Restarting daemon 2022-11-27T16:16:58.995 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-11-27T16:16:58.998 INFO:tasks.ceph.osd.2:Started 2022-11-27T16:16:58.998 INFO:tasks.ceph.osd.3:Restarting daemon 2022-11-27T16:16:58.998 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-11-27T16:16:59.001 INFO:tasks.ceph.osd.3:Started 2022-11-27T16:16:59.001 INFO:tasks.ceph.osd.4:Restarting daemon 2022-11-27T16:16:59.002 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-11-27T16:16:59.004 INFO:tasks.ceph.osd.4:Started 2022-11-27T16:16:59.004 INFO:tasks.ceph.osd.5:Restarting daemon 2022-11-27T16:16:59.005 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-11-27T16:16:59.007 INFO:tasks.ceph.osd.5:Started 2022-11-27T16:16:59.007 INFO:tasks.ceph.osd.6:Restarting daemon 2022-11-27T16:16:59.008 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-11-27T16:16:59.010 INFO:tasks.ceph.osd.6:Started 2022-11-27T16:16:59.010 INFO:tasks.ceph.osd.7:Restarting daemon 2022-11-27T16:16:59.011 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-11-27T16:16:59.013 INFO:tasks.ceph.osd.7:Started 2022-11-27T16:16:59.014 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:16:59.163 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.161+0000 7f4b0357d700 1 Processor -- start 2022-11-27T16:16:59.164 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.162+0000 7f4b0357d700 1 -- start start 2022-11-27T16:16:59.166 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.162+0000 7f4b0357d700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4afc159840 0x7f4afc0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:59.167 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.162+0000 7f4b0357d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4afc0af9d0 0x7f4afc0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:59.167 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.162+0000 7f4b0357d700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc0b0420 con 0x7f4afc0afe70 2022-11-27T16:16:59.167 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.162+0000 7f4b0357d700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc0b0700 con 0x7f4afc159840 2022-11-27T16:16:59.167 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.162+0000 7f4b0357d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc0b09e0 con 0x7f4afc0af9d0 2022-11-27T16:16:59.168 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.162+0000 7f4b01319700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4afc159840 0x7f4afc0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:59.168 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.162+0000 7f4b01b1a700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f4afc0afe70 0x7f4afc0acc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:55670/0 (socket says 172.21.15.134:55670) 2022-11-27T16:16:59.168 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4b01319700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4afc159840 0x7f4afc0b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37594/0 (socket says 172.21.15.134:37594) 2022-11-27T16:16:59.168 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4b01b1a700 1 -- 172.21.15.134:0/3070756913 learned_addr learned my addr 172.21.15.134:0/3070756913 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:16:59.169 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4b00b18700 1 --2- 172.21.15.134:0/3070756913 >> v2:172.21.15.191:3300/0 conn(0x7f4afc0af9d0 0x7f4afc0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:59.169 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4b01319700 1 -- 172.21.15.134:0/3070756913 >> v2:172.21.15.191:3300/0 conn(0x7f4afc0af9d0 msgr2=0x7f4afc0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:16:59.169 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4b01319700 1 --2- 172.21.15.134:0/3070756913 >> v2:172.21.15.191:3300/0 conn(0x7f4afc0af9d0 0x7f4afc0cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:59.169 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4b01319700 1 -- 172.21.15.134:0/3070756913 >> v1:172.21.15.134:6789/0 conn(0x7f4afc0afe70 legacy=0x7f4afc0acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:59.170 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4aebfff700 1 -- 172.21.15.134:0/3070756913 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265363347 0 0) 0x7f4afc0b0420 con 0x7f4afc0afe70 2022-11-27T16:16:59.170 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4b01319700 1 -- 172.21.15.134:0/3070756913 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0b0cc0 con 0x7f4afc159840 2022-11-27T16:16:59.170 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4b01319700 1 --2- 172.21.15.134:0/3070756913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc159840 0x7f4afc0b14c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4afc070fa0 tx=0x7f4af8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3422f045730dbd5e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:59.170 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4aebfff700 1 -- 172.21.15.134:0/3070756913 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8011830 con 0x7f4afc159840 2022-11-27T16:16:59.170 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4aebfff700 1 -- 172.21.15.134:0/3070756913 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af8011990 con 0x7f4afc159840 2022-11-27T16:16:59.171 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.163+0000 7f4aebfff700 1 -- 172.21.15.134:0/3070756913 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8011d20 con 0x7f4afc159840 2022-11-27T16:16:59.171 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.164+0000 7f4b0357d700 1 -- 172.21.15.134:0/3070756913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc159840 msgr2=0x7f4afc0b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:59.171 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.164+0000 7f4b0357d700 1 --2- 172.21.15.134:0/3070756913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc159840 0x7f4afc0b14c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4afc070fa0 tx=0x7f4af8009b20 comp rx=0 tx=0).stop 2022-11-27T16:16:59.171 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.164+0000 7f4b0357d700 1 -- 172.21.15.134:0/3070756913 shutdown_connections 2022-11-27T16:16:59.172 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.164+0000 7f4b0357d700 1 --2- 172.21.15.134:0/3070756913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc159840 0x7f4afc0b14c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:59.172 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.164+0000 7f4b0357d700 1 --2- 172.21.15.134:0/3070756913 >> v2:172.21.15.191:3300/0 conn(0x7f4afc0af9d0 0x7f4afc0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:59.172 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.164+0000 7f4b0357d700 1 -- 172.21.15.134:0/3070756913 >> 172.21.15.134:0/3070756913 conn(0x7f4afc069d10 msgr2=0x7f4afc099350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:59.173 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.164+0000 7f4b0357d700 1 -- 172.21.15.134:0/3070756913 shutdown_connections 2022-11-27T16:16:59.173 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.164+0000 7f4b0357d700 1 -- 172.21.15.134:0/3070756913 wait complete. 2022-11-27T16:16:59.176 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.165+0000 7f4b0357d700 1 Processor -- start 2022-11-27T16:16:59.177 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.165+0000 7f4b0357d700 1 -- start start 2022-11-27T16:16:59.177 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.166+0000 7f4b0357d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc0af9d0 0x7f4afc1f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:59.177 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.166+0000 7f4b0357d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4afc0afe70 0x7f4afc0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:59.177 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.166+0000 7f4b0357d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4afc159840 0x7f4afc0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:16:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.166+0000 7f4b0357d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0b31e0 con 0x7f4afc0af9d0 2022-11-27T16:16:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.166+0000 7f4b0357d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0c5600 con 0x7f4afc159840 2022-11-27T16:16:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.166+0000 7f4b0357d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0c6460 con 0x7f4afc0afe70 2022-11-27T16:16:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.166+0000 7f4b01319700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc0af9d0 0x7f4afc1f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:59.179 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b01319700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc0af9d0 0x7f4afc1f4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37600/0 (socket says 172.21.15.134:37600) 2022-11-27T16:16:59.179 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b01319700 1 -- 172.21.15.134:0/88461232 learned_addr learned my addr 172.21.15.134:0/88461232 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:16:59.179 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b01b1a700 1 --2- 172.21.15.134:0/88461232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4afc159840 0x7f4afc0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:59.179 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b00b18700 1 --2- 172.21.15.134:0/88461232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4afc0afe70 0x7f4afc0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:16:59.180 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b01319700 1 -- 172.21.15.134:0/88461232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4afc0afe70 msgr2=0x7f4afc0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:59.180 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b01319700 1 --2- 172.21.15.134:0/88461232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4afc0afe70 0x7f4afc0b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:59.180 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b01319700 1 -- 172.21.15.134:0/88461232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4afc159840 msgr2=0x7f4afc0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:59.180 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b01319700 1 --2- 172.21.15.134:0/88461232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4afc159840 0x7f4afc0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:59.180 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b01319700 1 -- 172.21.15.134:0/88461232 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af80094a0 con 0x7f4afc0af9d0 2022-11-27T16:16:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b01b1a700 1 --2- 172.21.15.134:0/88461232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4afc159840 0x7f4afc0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:16:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b01319700 1 --2- 172.21.15.134:0/88461232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc0af9d0 0x7f4afc1f4420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4af80090b0 tx=0x7f4af8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:16:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4ae9ffb700 1 -- 172.21.15.134:0/88461232 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af80036a0 con 0x7f4afc0af9d0 2022-11-27T16:16:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4ae9ffb700 1 -- 172.21.15.134:0/88461232 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af801e070 con 0x7f4afc0af9d0 2022-11-27T16:16:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b0357d700 1 -- 172.21.15.134:0/88461232 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc0c5850 con 0x7f4afc0af9d0 2022-11-27T16:16:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4ae9ffb700 1 -- 172.21.15.134:0/88461232 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8003a20 con 0x7f4afc0af9d0 2022-11-27T16:16:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.167+0000 7f4b0357d700 1 -- 172.21.15.134:0/88461232 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc0c5e10 con 0x7f4afc0af9d0 2022-11-27T16:16:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.168+0000 7f4b0357d700 1 -- 172.21.15.134:0/88461232 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc0b5de0 con 0x7f4afc0af9d0 2022-11-27T16:16:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.168+0000 7f4ae9ffb700 1 -- 172.21.15.134:0/88461232 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f4af800f050 con 0x7f4afc0af9d0 2022-11-27T16:16:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.168+0000 7f4ae9ffb700 1 -- 172.21.15.134:0/88461232 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f4af8024400 con 0x7f4afc0af9d0 2022-11-27T16:16:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.170+0000 7f4ae9ffb700 1 -- 172.21.15.134:0/88461232 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f4af8032060 con 0x7f4afc0af9d0 2022-11-27T16:16:59.247 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.239+0000 7f4b0357d700 1 -- 172.21.15.134:0/88461232 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4afc0cea60 con 0x7f4afc0af9d0 2022-11-27T16:16:59.248 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.240+0000 7f4ae9ffb700 1 -- 172.21.15.134:0/88461232 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f4af8020070 con 0x7f4afc0af9d0 2022-11-27T16:16:59.248 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:16:59.249 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":10,"fsid":"34bf4849-eb1e-4259-a7f0-0be8a82c33c9","created":"2022-11-27T16:16:56.408900+0000","modified":"2022-11-27T16:16:58.964080+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T16:16:58.964080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"f44064b8-6282-45ff-94ab-f3a2eb231dc1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"af1b8ac9-0335-4550-aaf8-9f15545625e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5bbdd443-3ec5-40fb-86af-3e5a6ab1606a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"795fd2fb-0e2e-4419-9b54-ea28ba0ee835","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"36005e5e-73a9-4d10-aba3-480165df1594","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6f5b5e55-e69b-4097-a47c-cd7fc0681dbd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"074077c9-fc46-4f79-8368-5a96eba6da06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"112889bf-b637-4500-a30d-03e23e8a3267","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:16:59.251 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.241+0000 7f4b0357d700 1 -- 172.21.15.134:0/88461232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc0af9d0 msgr2=0x7f4afc1f4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:16:59.252 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.241+0000 7f4b0357d700 1 --2- 172.21.15.134:0/88461232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc0af9d0 0x7f4afc1f4420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4af80090b0 tx=0x7f4af8004750 comp rx=0 tx=0).stop 2022-11-27T16:16:59.252 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.241+0000 7f4b0357d700 1 -- 172.21.15.134:0/88461232 shutdown_connections 2022-11-27T16:16:59.252 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.241+0000 7f4b0357d700 1 --2- 172.21.15.134:0/88461232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4afc159840 0x7f4afc0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:59.253 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.241+0000 7f4b0357d700 1 --2- 172.21.15.134:0/88461232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4afc0af9d0 0x7f4afc1f4420 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:59.253 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.241+0000 7f4b0357d700 1 --2- 172.21.15.134:0/88461232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4afc0afe70 0x7f4afc0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:16:59.253 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.241+0000 7f4b0357d700 1 -- 172.21.15.134:0/88461232 >> 172.21.15.134:0/88461232 conn(0x7f4afc069d10 msgr2=0x7f4afc1587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:16:59.254 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.241+0000 7f4b0357d700 1 -- 172.21.15.134:0/88461232 shutdown_connections 2022-11-27T16:16:59.254 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:16:59.241+0000 7f4b0357d700 1 -- 172.21.15.134:0/88461232 wait complete. 2022-11-27T16:16:59.256 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-11-27T16:16:59.256 INFO:tasks.ceph:Waiting for OSDs to come up 2022-11-27T16:16:59.359 INFO:tasks.ceph.osd.1.smithi134.stderr:2022-11-27T16:16:59.357+0000 7f74a2d315c0 -1 Falling back to public interface 2022-11-27T16:16:59.407 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:59.405+0000 7f75e0047200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:16:59.469 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:59.467+0000 7f46006a3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:16:59.483 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:59.482+0000 7f75e0047200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T16:16:59.544 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:59.543+0000 7f46006a3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T16:16:59.608 INFO:tasks.ceph.osd.0.smithi134.stderr:2022-11-27T16:16:59.606+0000 7f4ab86b45c0 -1 Falling back to public interface 2022-11-27T16:16:59.618 INFO:tasks.ceph.osd.5.smithi191.stderr:2022-11-27T16:16:59.616+0000 7f115f0685c0 -1 Falling back to public interface 2022-11-27T16:16:59.619 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:59.617+0000 7f75e0047200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:16:59.621 INFO:tasks.ceph.osd.7.smithi191.stderr:2022-11-27T16:16:59.619+0000 7fd06d2465c0 -1 Falling back to public interface 2022-11-27T16:16:59.621 INFO:tasks.ceph.osd.2.smithi134.stderr:2022-11-27T16:16:59.619+0000 7fe06a83a5c0 -1 Falling back to public interface 2022-11-27T16:16:59.625 INFO:tasks.ceph.osd.3.smithi134.stderr:2022-11-27T16:16:59.623+0000 7f53e9d955c0 -1 Falling back to public interface 2022-11-27T16:16:59.628 INFO:tasks.ceph.osd.4.smithi191.stderr:2022-11-27T16:16:59.626+0000 7f260bba25c0 -1 Falling back to public interface 2022-11-27T16:16:59.631 INFO:tasks.ceph.osd.6.smithi191.stderr:2022-11-27T16:16:59.629+0000 7f23d754d5c0 -1 Falling back to public interface 2022-11-27T16:16:59.683 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:59.681+0000 7f46006a3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:16:59.777 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:59.776+0000 7f75e0047200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T16:16:59.839 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:59.838+0000 7f46006a3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T16:16:59.908 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:16:59.906+0000 7f75e0047200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T16:16:59.971 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:16:59.969+0000 7f46006a3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T16:17:00.061 DEBUG:teuthology.orchestra.run.smithi134:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T16:17:00.175 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.173+0000 7fe13f457700 1 Processor -- start 2022-11-27T16:17:00.175 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.173+0000 7fe13f457700 1 -- start start 2022-11-27T16:17:00.177 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13f457700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1380cf150 0x7fe1380b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:00.178 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13f457700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe1380d0850 0x7fe1380b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:00.178 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13f457700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1380ce320 con 0x7fe1380b34c0 2022-11-27T16:17:00.178 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13f457700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1380b54d0 con 0x7fe1380d0850 2022-11-27T16:17:00.179 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13f457700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380cda00 con 0x7fe1380cf150 2022-11-27T16:17:00.179 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13d9f4700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe1380d0850 0x7fe1380b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:00.179 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13d9f4700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe1380d0850 0x7fe1380b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37610/0 (socket says 172.21.15.134:37610) 2022-11-27T16:17:00.180 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13d9f4700 1 -- 172.21.15.134:0/571081183 learned_addr learned my addr 172.21.15.134:0/571081183 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:00.180 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13c9f2700 1 --2- 172.21.15.134:0/571081183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1380cf150 0x7fe1380b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:00.180 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13d9f4700 1 -- 172.21.15.134:0/571081183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1380cf150 msgr2=0x7fe1380b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:00.180 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13d9f4700 1 --2- 172.21.15.134:0/571081183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1380cf150 0x7fe1380b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:00.181 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13d9f4700 1 -- 172.21.15.134:0/571081183 >> v1:172.21.15.191:6789/0 conn(0x7fe1380b34c0 legacy=0x7fe1380cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:00.181 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.174+0000 7fe13d9f4700 1 -- 172.21.15.134:0/571081183 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1380cdce0 con 0x7fe1380d0850 2022-11-27T16:17:00.181 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe13d9f4700 1 --2- 172.21.15.134:0/571081183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 0x7fe1380b5ea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe130009e50 tx=0x7fe13000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c747fb18f04fbd21 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:00.181 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe127fff700 1 -- 172.21.15.134:0/571081183 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13000e070 con 0x7fe1380d0850 2022-11-27T16:17:00.181 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe127fff700 1 -- 172.21.15.134:0/571081183 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13000b910 con 0x7fe1380d0850 2022-11-27T16:17:00.182 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe127fff700 1 -- 172.21.15.134:0/571081183 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13000bcc0 con 0x7fe1380d0850 2022-11-27T16:17:00.182 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe13f457700 1 -- 172.21.15.134:0/571081183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 msgr2=0x7fe1380b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:00.182 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe13f457700 1 --2- 172.21.15.134:0/571081183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 0x7fe1380b5ea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe130009e50 tx=0x7fe13000b330 comp rx=0 tx=0).stop 2022-11-27T16:17:00.182 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe13f457700 1 -- 172.21.15.134:0/571081183 shutdown_connections 2022-11-27T16:17:00.183 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe13f457700 1 --2- 172.21.15.134:0/571081183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 0x7fe1380b5ea0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:00.183 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe13f457700 1 --2- 172.21.15.134:0/571081183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1380cf150 0x7fe1380b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:00.183 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe13f457700 1 -- 172.21.15.134:0/571081183 >> 172.21.15.134:0/571081183 conn(0x7fe13809b480 msgr2=0x7fe1380d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:00.183 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.175+0000 7fe13f457700 1 -- 172.21.15.134:0/571081183 shutdown_connections 2022-11-27T16:17:00.184 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.176+0000 7fe13f457700 1 -- 172.21.15.134:0/571081183 wait complete. 2022-11-27T16:17:00.187 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.177+0000 7fe13f457700 1 Processor -- start 2022-11-27T16:17:00.187 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.177+0000 7fe13f457700 1 -- start start 2022-11-27T16:17:00.188 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.177+0000 7fe13f457700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1380b34c0 0x7fe13814c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:00.188 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.177+0000 7fe13f457700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1380cf150 0x7fe1380c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:00.188 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.177+0000 7fe13f457700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 0x7fe13809bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:00.188 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.177+0000 7fe13f457700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13815da80 con 0x7fe1380d0850 2022-11-27T16:17:00.189 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.177+0000 7fe13f457700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13815dd60 con 0x7fe1380b34c0 2022-11-27T16:17:00.189 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.177+0000 7fe13f457700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13815e040 con 0x7fe1380cf150 2022-11-27T16:17:00.189 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.177+0000 7fe13d9f4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 0x7fe13809bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:00.189 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13d9f4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 0x7fe13809bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37616/0 (socket says 172.21.15.134:37616) 2022-11-27T16:17:00.190 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13d9f4700 1 -- 172.21.15.134:0/17995997 learned_addr learned my addr 172.21.15.134:0/17995997 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:00.190 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13c9f2700 1 --2- 172.21.15.134:0/17995997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1380cf150 0x7fe1380c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:00.190 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13d1f3700 1 --2- 172.21.15.134:0/17995997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1380b34c0 0x7fe13814c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:00.190 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13d9f4700 1 -- 172.21.15.134:0/17995997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1380cf150 msgr2=0x7fe1380c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:00.191 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13d9f4700 1 --2- 172.21.15.134:0/17995997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1380cf150 0x7fe1380c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:00.191 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13d9f4700 1 -- 172.21.15.134:0/17995997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1380b34c0 msgr2=0x7fe13814c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:00.191 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13d9f4700 1 --2- 172.21.15.134:0/17995997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1380b34c0 0x7fe13814c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:00.191 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13d9f4700 1 -- 172.21.15.134:0/17995997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130009b30 con 0x7fe1380d0850 2022-11-27T16:17:00.192 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13d9f4700 1 --2- 172.21.15.134:0/17995997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 0x7fe13809bef0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe130012000 tx=0x7fe130004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:00.192 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe125ffb700 1 -- 172.21.15.134:0/17995997 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130004840 con 0x7fe1380d0850 2022-11-27T16:17:00.192 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe125ffb700 1 -- 172.21.15.134:0/17995997 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1300099b0 con 0x7fe1380d0850 2022-11-27T16:17:00.192 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13f457700 1 -- 172.21.15.134:0/17995997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13815e320 con 0x7fe1380d0850 2022-11-27T16:17:00.192 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe125ffb700 1 -- 172.21.15.134:0/17995997 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13001c680 con 0x7fe1380d0850 2022-11-27T16:17:00.193 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe13f457700 1 -- 172.21.15.134:0/17995997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1380c9880 con 0x7fe1380d0850 2022-11-27T16:17:00.193 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe125ffb700 1 -- 172.21.15.134:0/17995997 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fe13002c070 con 0x7fe1380d0850 2022-11-27T16:17:00.193 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.178+0000 7fe125ffb700 1 -- 172.21.15.134:0/17995997 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fe13001cdd0 con 0x7fe1380d0850 2022-11-27T16:17:00.193 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.179+0000 7fe13f457700 1 -- 172.21.15.134:0/17995997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1380d28a0 con 0x7fe1380d0850 2022-11-27T16:17:00.194 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.181+0000 7fe125ffb700 1 -- 172.21.15.134:0/17995997 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fe1300446d0 con 0x7fe1380d0850 2022-11-27T16:17:00.252 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.251+0000 7fe13f457700 1 -- 172.21.15.134:0/17995997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe13815e840 con 0x7fe1380d0850 2022-11-27T16:17:00.253 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.251+0000 7fe125ffb700 1 -- 172.21.15.134:0/17995997 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7fe13815e840 con 0x7fe1380d0850 2022-11-27T16:17:00.254 INFO:teuthology.misc.health.smithi134.stdout: 2022-11-27T16:17:00.254 INFO:teuthology.misc.health.smithi134.stdout:{"epoch":10,"fsid":"34bf4849-eb1e-4259-a7f0-0be8a82c33c9","created":"2022-11-27T16:16:56.408900+0000","modified":"2022-11-27T16:16:58.964080+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T16:16:58.964080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"f44064b8-6282-45ff-94ab-f3a2eb231dc1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"af1b8ac9-0335-4550-aaf8-9f15545625e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5bbdd443-3ec5-40fb-86af-3e5a6ab1606a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"795fd2fb-0e2e-4419-9b54-ea28ba0ee835","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"36005e5e-73a9-4d10-aba3-480165df1594","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6f5b5e55-e69b-4097-a47c-cd7fc0681dbd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"074077c9-fc46-4f79-8368-5a96eba6da06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"112889bf-b637-4500-a30d-03e23e8a3267","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:17:00.255 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.252+0000 7fe13f457700 1 -- 172.21.15.134:0/17995997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 msgr2=0x7fe13809bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:00.255 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.252+0000 7fe13f457700 1 --2- 172.21.15.134:0/17995997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 0x7fe13809bef0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe130012000 tx=0x7fe130004ad0 comp rx=0 tx=0).stop 2022-11-27T16:17:00.256 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.252+0000 7fe13f457700 1 -- 172.21.15.134:0/17995997 shutdown_connections 2022-11-27T16:17:00.256 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.252+0000 7fe13f457700 1 --2- 172.21.15.134:0/17995997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1380d0850 0x7fe13809bef0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:00.256 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.252+0000 7fe13f457700 1 --2- 172.21.15.134:0/17995997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1380b34c0 0x7fe13814c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:00.256 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.252+0000 7fe13f457700 1 --2- 172.21.15.134:0/17995997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1380cf150 0x7fe1380c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:00.257 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.252+0000 7fe13f457700 1 -- 172.21.15.134:0/17995997 >> 172.21.15.134:0/17995997 conn(0x7fe13809b480 msgr2=0x7fe1380ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:00.257 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.252+0000 7fe13f457700 1 -- 172.21.15.134:0/17995997 shutdown_connections 2022-11-27T16:17:00.257 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:00.252+0000 7fe13f457700 1 -- 172.21.15.134:0/17995997 wait complete. 2022-11-27T16:17:00.265 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-11-27T16:17:00.266 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:17:00.264+0000 7f75e0047200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T16:17:00.320 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:17:00.318+0000 7f46006a3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T16:17:00.417 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:17:00.415+0000 7f75e0047200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T16:17:00.472 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:17:00.471+0000 7f46006a3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T16:17:00.533 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:17:00.531+0000 7f75e0047200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T16:17:00.588 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:17:00.586+0000 7f46006a3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T16:17:00.675 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:17:00.673+0000 7f75e0047200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T16:17:00.731 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:17:00.729+0000 7f46006a3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T16:17:00.774 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:17:00.772+0000 7f75e0047200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T16:17:00.830 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:17:00.828+0000 7f46006a3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T16:17:01.128 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:17:01.126+0000 7f75e0047200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T16:17:01.185 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:17:01.183+0000 7f46006a3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T16:17:01.702 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:17:01.701+0000 7f75e0047200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T16:17:01.763 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-11-27T16:17:01.761+0000 7f46006a3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T16:17:02.730 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:17:02.728+0000 7f7598411700 -1 mgr.server handle_report got status from non-daemon mon.c 2022-11-27T16:17:02.730 INFO:tasks.ceph.mgr.y.smithi134.stderr:2022-11-27T16:17:02.729+0000 7f7598411700 -1 mgr.server handle_report got status from non-daemon mon.b 2022-11-27T16:17:02.902 INFO:tasks.ceph.osd.1.smithi134.stderr:2022-11-27T16:17:02.901+0000 7f74a2d315c0 -1 osd.1 0 log_to_monitors true 2022-11-27T16:17:03.153 INFO:tasks.ceph.osd.7.smithi191.stderr:2022-11-27T16:17:03.151+0000 7fd06d2465c0 -1 osd.7 0 log_to_monitors true 2022-11-27T16:17:03.164 INFO:tasks.ceph.osd.0.smithi134.stderr:2022-11-27T16:17:03.163+0000 7f4ab86b45c0 -1 osd.0 0 log_to_monitors true 2022-11-27T16:17:03.168 INFO:tasks.ceph.osd.3.smithi134.stderr:2022-11-27T16:17:03.166+0000 7f53e9d955c0 -1 osd.3 0 log_to_monitors true 2022-11-27T16:17:03.170 INFO:tasks.ceph.osd.2.smithi134.stderr:2022-11-27T16:17:03.168+0000 7fe06a83a5c0 -1 osd.2 0 log_to_monitors true 2022-11-27T16:17:03.183 INFO:tasks.ceph.osd.5.smithi191.stderr:2022-11-27T16:17:03.182+0000 7f115f0685c0 -1 osd.5 0 log_to_monitors true 2022-11-27T16:17:03.193 INFO:tasks.ceph.osd.6.smithi191.stderr:2022-11-27T16:17:03.192+0000 7f23d754d5c0 -1 osd.6 0 log_to_monitors true 2022-11-27T16:17:03.207 INFO:tasks.ceph.osd.4.smithi191.stderr:2022-11-27T16:17:03.205+0000 7f260bba25c0 -1 osd.4 0 log_to_monitors true 2022-11-27T16:17:06.757 INFO:tasks.ceph.osd.5.smithi191.stderr:2022-11-27T16:17:06.751+0000 7f1152c1f700 -1 osd.5 0 waiting for initial osdmap 2022-11-27T16:17:06.758 INFO:tasks.ceph.osd.3.smithi134.stderr:2022-11-27T16:17:06.756+0000 7f53dd94c700 -1 osd.3 0 waiting for initial osdmap 2022-11-27T16:17:06.770 INFO:tasks.ceph.osd.3.smithi134.stderr:2022-11-27T16:17:06.769+0000 7f53d5d27700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:17:06.771 INFO:tasks.ceph.osd.5.smithi191.stderr:2022-11-27T16:17:06.770+0000 7f114affa700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:17:06.773 INFO:tasks.ceph.osd.6.smithi191.stderr:2022-11-27T16:17:06.772+0000 7f23cb104700 -1 osd.6 0 waiting for initial osdmap 2022-11-27T16:17:06.786 INFO:tasks.ceph.osd.6.smithi191.stderr:2022-11-27T16:17:06.785+0000 7f23c34df700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:17:06.792 INFO:tasks.ceph.osd.2.smithi134.stderr:2022-11-27T16:17:06.788+0000 7fe05e3f1700 -1 osd.2 0 waiting for initial osdmap 2022-11-27T16:17:06.799 INFO:tasks.ceph.osd.7.smithi191.stderr:2022-11-27T16:17:06.797+0000 7fd060dfd700 -1 osd.7 0 waiting for initial osdmap 2022-11-27T16:17:06.799 INFO:tasks.ceph.osd.0.smithi134.stderr:2022-11-27T16:17:06.798+0000 7f4aac26b700 -1 osd.0 0 waiting for initial osdmap 2022-11-27T16:17:06.803 INFO:tasks.ceph.osd.1.smithi134.stderr:2022-11-27T16:17:06.801+0000 7f74968e8700 -1 osd.1 0 waiting for initial osdmap 2022-11-27T16:17:06.804 INFO:tasks.ceph.osd.2.smithi134.stderr:2022-11-27T16:17:06.803+0000 7fe0567cc700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:17:06.806 INFO:tasks.ceph.osd.4.smithi191.stderr:2022-11-27T16:17:06.804+0000 7f25ff759700 -1 osd.4 0 waiting for initial osdmap 2022-11-27T16:17:06.812 INFO:tasks.ceph.osd.0.smithi134.stderr:2022-11-27T16:17:06.811+0000 7f4aa4646700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:17:06.816 INFO:tasks.ceph.osd.1.smithi134.stderr:2022-11-27T16:17:06.815+0000 7f748ecc3700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:17:06.819 INFO:tasks.ceph.osd.7.smithi191.stderr:2022-11-27T16:17:06.819+0000 7fd0591d8700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:17:06.823 INFO:tasks.ceph.osd.4.smithi191.stderr:2022-11-27T16:17:06.822+0000 7f25f7b34700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:17:07.073 DEBUG:teuthology.orchestra.run.smithi134:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T16:17:07.186 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.184+0000 7f227916d700 1 Processor -- start 2022-11-27T16:17:07.186 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.184+0000 7f227916d700 1 -- start start 2022-11-27T16:17:07.189 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227916d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f22741588f0 0x7f22740709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:07.190 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227916d700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f22740cb6f0 0x7f22740cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:07.190 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227916d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740ccc20 con 0x7f22740c3d10 2022-11-27T16:17:07.190 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227916d700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2274070f20 con 0x7f22740cb6f0 2022-11-27T16:17:07.191 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227916d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22740cc080 con 0x7f22741588f0 2022-11-27T16:17:07.191 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227259c700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f22740cb6f0 0x7f22740cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:07.191 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227259c700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f22740cb6f0 0x7f22740cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37716/0 (socket says 172.21.15.134:37716) 2022-11-27T16:17:07.192 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227259c700 1 -- 172.21.15.134:0/3175097314 learned_addr learned my addr 172.21.15.134:0/3175097314 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:07.192 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f2272d9d700 1 --2- 172.21.15.134:0/3175097314 >> v2:172.21.15.191:3300/0 conn(0x7f22741588f0 0x7f22740709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:07.192 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227259c700 1 -- 172.21.15.134:0/3175097314 >> v2:172.21.15.191:3300/0 conn(0x7f22741588f0 msgr2=0x7f22740709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:07.192 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227259c700 1 --2- 172.21.15.134:0/3175097314 >> v2:172.21.15.191:3300/0 conn(0x7f22741588f0 0x7f22740709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:07.192 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227259c700 1 -- 172.21.15.134:0/3175097314 >> v1:172.21.15.191:6789/0 conn(0x7f22740c3d10 legacy=0x7f22740c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:07.193 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227259c700 1 -- 172.21.15.134:0/3175097314 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274158cc0 con 0x7f22740cb6f0 2022-11-27T16:17:07.193 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f227259c700 1 --2- 172.21.15.134:0/3175097314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740cb6f0 0x7f22740cbb40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2274071470 tx=0x7f2264009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0eff27deea9a57 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:07.193 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f2271d9b700 1 -- 172.21.15.134:0/3175097314 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264011830 con 0x7f22740cb6f0 2022-11-27T16:17:07.193 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f2271d9b700 1 -- 172.21.15.134:0/3175097314 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2264011990 con 0x7f22740cb6f0 2022-11-27T16:17:07.194 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.185+0000 7f2271d9b700 1 -- 172.21.15.134:0/3175097314 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264011d20 con 0x7f22740cb6f0 2022-11-27T16:17:07.194 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.186+0000 7f227916d700 1 -- 172.21.15.134:0/3175097314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740cb6f0 msgr2=0x7f22740cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:07.194 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.186+0000 7f227916d700 1 --2- 172.21.15.134:0/3175097314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740cb6f0 0x7f22740cbb40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2274071470 tx=0x7f2264009bd0 comp rx=0 tx=0).stop 2022-11-27T16:17:07.194 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.186+0000 7f227916d700 1 -- 172.21.15.134:0/3175097314 shutdown_connections 2022-11-27T16:17:07.195 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.186+0000 7f227916d700 1 --2- 172.21.15.134:0/3175097314 >> v2:172.21.15.191:3300/0 conn(0x7f22741588f0 0x7f22740709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:07.195 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.186+0000 7f227916d700 1 --2- 172.21.15.134:0/3175097314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740cb6f0 0x7f22740cbb40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:07.195 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.186+0000 7f227916d700 1 -- 172.21.15.134:0/3175097314 >> 172.21.15.134:0/3175097314 conn(0x7f22740b0aa0 msgr2=0x7f22740b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:07.195 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.186+0000 7f227916d700 1 -- 172.21.15.134:0/3175097314 shutdown_connections 2022-11-27T16:17:07.195 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.186+0000 7f227916d700 1 -- 172.21.15.134:0/3175097314 wait complete. 2022-11-27T16:17:07.196 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f227916d700 1 Processor -- start 2022-11-27T16:17:07.196 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f227916d700 1 -- start start 2022-11-27T16:17:07.200 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f227916d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740c3d10 0x7f22741486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:07.201 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f227916d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22740cb6f0 0x7f227415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:07.201 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f227916d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741588f0 0x7f227415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:07.202 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f227916d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22740ac330 con 0x7f22740c3d10 2022-11-27T16:17:07.202 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f227916d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227415bca0 con 0x7f22741588f0 2022-11-27T16:17:07.203 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f227916d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22740aba00 con 0x7f22740cb6f0 2022-11-27T16:17:07.203 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f2272d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740c3d10 0x7f22741486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:07.203 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f2272d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740c3d10 0x7f22741486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37720/0 (socket says 172.21.15.134:37720) 2022-11-27T16:17:07.204 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.188+0000 7f2272d9d700 1 -- 172.21.15.134:0/1837965605 learned_addr learned my addr 172.21.15.134:0/1837965605 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:07.204 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f227259c700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22740cb6f0 0x7f227415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:07.205 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f227359e700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741588f0 0x7f227415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:07.205 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f2272d9d700 1 -- 172.21.15.134:0/1837965605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22740cb6f0 msgr2=0x7f227415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:07.206 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f2272d9d700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22740cb6f0 0x7f227415b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:07.206 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f2272d9d700 1 -- 172.21.15.134:0/1837965605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741588f0 msgr2=0x7f227415b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:07.206 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f2272d9d700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741588f0 0x7f227415b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:07.207 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f2272d9d700 1 -- 172.21.15.134:0/1837965605 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009550 con 0x7f22740c3d10 2022-11-27T16:17:07.207 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f2272d9d700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740c3d10 0x7f22741486f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f226800e870 tx=0x7f226800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:07.208 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f225b7fe700 1 -- 172.21.15.134:0/1837965605 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268009590 con 0x7f22740c3d10 2022-11-27T16:17:07.208 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f225b7fe700 1 -- 172.21.15.134:0/1837965605 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268013680 con 0x7f22740c3d10 2022-11-27T16:17:07.209 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f227916d700 1 -- 172.21.15.134:0/1837965605 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22740abce0 con 0x7f22740c3d10 2022-11-27T16:17:07.209 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f225b7fe700 1 -- 172.21.15.134:0/1837965605 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22680139f0 con 0x7f22740c3d10 2022-11-27T16:17:07.209 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.189+0000 7f227916d700 1 -- 172.21.15.134:0/1837965605 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22740cb0c0 con 0x7f22740c3d10 2022-11-27T16:17:07.210 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.190+0000 7f227916d700 1 -- 172.21.15.134:0/1837965605 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2274073030 con 0x7f22740c3d10 2022-11-27T16:17:07.210 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.190+0000 7f225b7fe700 1 -- 172.21.15.134:0/1837965605 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7f2268019030 con 0x7f22740c3d10 2022-11-27T16:17:07.210 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.191+0000 7f225b7fe700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f2254066540 0x7f22540689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:07.211 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.191+0000 7f225b7fe700 1 -- 172.21.15.134:0/1837965605 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7f2268082530 con 0x7f22740c3d10 2022-11-27T16:17:07.211 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.191+0000 7f227259c700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f2254066540 0x7f22540689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:07.211 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.191+0000 7f227259c700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f2254066540 0x7f22540689f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2264009c10 tx=0x7f2264008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:07.211 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.195+0000 7f225b7fe700 1 -- 172.21.15.134:0/1837965605 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f2268050b80 con 0x7f22740c3d10 2022-11-27T16:17:07.377 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.375+0000 7f227916d700 1 -- 172.21.15.134:0/1837965605 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f227406ba10 con 0x7f22740c3d10 2022-11-27T16:17:07.377 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.375+0000 7f225b7fe700 1 -- 172.21.15.134:0/1837965605 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f22680541a0 con 0x7f22740c3d10 2022-11-27T16:17:07.378 INFO:teuthology.misc.health.smithi134.stdout: 2022-11-27T16:17:07.378 INFO:teuthology.misc.health.smithi134.stdout:{"epoch":12,"fsid":"34bf4849-eb1e-4259-a7f0-0be8a82c33c9","created":"2022-11-27T16:16:56.408900+0000","modified":"2022-11-27T16:17:04.731433+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T16:16:58.964080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"f44064b8-6282-45ff-94ab-f3a2eb231dc1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"af1b8ac9-0335-4550-aaf8-9f15545625e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5bbdd443-3ec5-40fb-86af-3e5a6ab1606a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"795fd2fb-0e2e-4419-9b54-ea28ba0ee835","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"36005e5e-73a9-4d10-aba3-480165df1594","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6f5b5e55-e69b-4097-a47c-cd7fc0681dbd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"074077c9-fc46-4f79-8368-5a96eba6da06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"112889bf-b637-4500-a30d-03e23e8a3267","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:17:07.380 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.377+0000 7f227916d700 1 -- 172.21.15.134:0/1837965605 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f2254066540 msgr2=0x7f22540689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:07.380 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f2254066540 0x7f22540689f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2264009c10 tx=0x7f2264008030 comp rx=0 tx=0).stop 2022-11-27T16:17:07.381 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 -- 172.21.15.134:0/1837965605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740c3d10 msgr2=0x7f22741486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:07.381 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740c3d10 0x7f22741486f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f226800e870 tx=0x7f226800eb80 comp rx=0 tx=0).stop 2022-11-27T16:17:07.381 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 -- 172.21.15.134:0/1837965605 shutdown_connections 2022-11-27T16:17:07.381 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f2254066540 0x7f22540689f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:07.382 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22741588f0 0x7f227415b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:07.382 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22740c3d10 0x7f22741486f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:07.382 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 --2- 172.21.15.134:0/1837965605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22740cb6f0 0x7f227415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:07.382 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 -- 172.21.15.134:0/1837965605 >> 172.21.15.134:0/1837965605 conn(0x7f22740b0aa0 msgr2=0x7f2274152f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:07.383 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 -- 172.21.15.134:0/1837965605 shutdown_connections 2022-11-27T16:17:07.383 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:07.378+0000 7f227916d700 1 -- 172.21.15.134:0/1837965605 wait complete. 2022-11-27T16:17:07.391 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-11-27T16:17:14.199 DEBUG:teuthology.orchestra.run.smithi134:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T16:17:14.322 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.320+0000 7f90e720b700 1 Processor -- start 2022-11-27T16:17:14.323 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.320+0000 7f90e720b700 1 -- start start 2022-11-27T16:17:14.326 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90e720b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 0x7f90e00707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.326 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90e720b700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90e00ce7d0 0x7f90e00693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.326 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90e720b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f90e0069930 0x7f90e015f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.326 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90e720b700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e0069db0 con 0x7f90e00ce7d0 2022-11-27T16:17:14.327 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90e720b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e00cc700 con 0x7f90e0069930 2022-11-27T16:17:14.327 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90e720b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00cb6f0 con 0x7f90e015b9b0 2022-11-27T16:17:14.327 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90dffff700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90e00ce7d0 0x7f90e00693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.327 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90dffff700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90e00ce7d0 0x7f90e00693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37946/0 (socket says 172.21.15.134:37946) 2022-11-27T16:17:14.328 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90dffff700 1 -- 172.21.15.134:0/1415123444 learned_addr learned my addr 172.21.15.134:0/1415123444 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:14.328 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90e57a8700 1 --2- 172.21.15.134:0/1415123444 >> v2:172.21.15.191:3300/0 conn(0x7f90e0069930 0x7f90e015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.328 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.321+0000 7f90e4fa7700 1 --2- 172.21.15.134:0/1415123444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 0x7f90e00707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.328 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90dffff700 1 -- 172.21.15.134:0/1415123444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 msgr2=0x7f90e00707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:14.329 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90dffff700 1 --2- 172.21.15.134:0/1415123444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 0x7f90e00707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.329 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90dffff700 1 -- 172.21.15.134:0/1415123444 >> v2:172.21.15.191:3300/0 conn(0x7f90e0069930 msgr2=0x7f90e015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:14.329 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90dffff700 1 --2- 172.21.15.134:0/1415123444 >> v2:172.21.15.191:3300/0 conn(0x7f90e0069930 0x7f90e015f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.329 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90dffff700 1 -- 172.21.15.134:0/1415123444 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e00cb940 con 0x7f90e00ce7d0 2022-11-27T16:17:14.329 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90e57a8700 1 --2- 172.21.15.134:0/1415123444 >> v2:172.21.15.191:3300/0 conn(0x7f90e0069930 0x7f90e015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:17:14.330 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90e4fa7700 1 --2- 172.21.15.134:0/1415123444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 0x7f90e00707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:17:14.330 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90dffff700 1 --2- 172.21.15.134:0/1415123444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 0x7f90e00693f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f90e006ea60 tx=0x7f90cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07cae30f1308173 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:14.330 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90df7fe700 1 -- 172.21.15.134:0/1415123444 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc00b870 con 0x7f90e00ce7d0 2022-11-27T16:17:14.330 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90df7fe700 1 -- 172.21.15.134:0/1415123444 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc00b9d0 con 0x7f90e00ce7d0 2022-11-27T16:17:14.331 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90df7fe700 1 -- 172.21.15.134:0/1415123444 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc00bd60 con 0x7f90e00ce7d0 2022-11-27T16:17:14.331 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90e720b700 1 -- 172.21.15.134:0/1415123444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 msgr2=0x7f90e00693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:14.331 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90e720b700 1 --2- 172.21.15.134:0/1415123444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 0x7f90e00693f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f90e006ea60 tx=0x7f90cc009bd0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.331 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90e720b700 1 -- 172.21.15.134:0/1415123444 shutdown_connections 2022-11-27T16:17:14.332 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.322+0000 7f90e720b700 1 --2- 172.21.15.134:0/1415123444 >> v2:172.21.15.191:3300/0 conn(0x7f90e0069930 0x7f90e015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.332 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.323+0000 7f90e720b700 1 --2- 172.21.15.134:0/1415123444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 0x7f90e00707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.332 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.323+0000 7f90e720b700 1 --2- 172.21.15.134:0/1415123444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 0x7f90e00693f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.332 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.323+0000 7f90e720b700 1 -- 172.21.15.134:0/1415123444 >> 172.21.15.134:0/1415123444 conn(0x7f90e00b0aa0 msgr2=0x7f90e00b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:14.333 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.323+0000 7f90e720b700 1 -- 172.21.15.134:0/1415123444 shutdown_connections 2022-11-27T16:17:14.333 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.323+0000 7f90e720b700 1 -- 172.21.15.134:0/1415123444 wait complete. 2022-11-27T16:17:14.333 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.324+0000 7f90e720b700 1 Processor -- start 2022-11-27T16:17:14.333 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.324+0000 7f90e720b700 1 -- start start 2022-11-27T16:17:14.336 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90e720b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90e0069930 0x7f90e014ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.337 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90e720b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 0x7f90e006db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.337 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90e720b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 0x7f90e0149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.337 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90e720b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e014ffb0 con 0x7f90e00ce7d0 2022-11-27T16:17:14.337 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90e720b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00b49d0 con 0x7f90e0069930 2022-11-27T16:17:14.338 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90e720b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e014f430 con 0x7f90e015b9b0 2022-11-27T16:17:14.338 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 0x7f90e006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.338 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 0x7f90e006db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37954/0 (socket says 172.21.15.134:37954) 2022-11-27T16:17:14.338 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dffff700 1 -- 172.21.15.134:0/1418350955 learned_addr learned my addr 172.21.15.134:0/1418350955 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:14.339 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90e4fa7700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90e0069930 0x7f90e014ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.339 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90e57a8700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 0x7f90e0149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.339 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dffff700 1 -- 172.21.15.134:0/1418350955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 msgr2=0x7f90e0149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:14.339 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dffff700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 0x7f90e0149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.340 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dffff700 1 -- 172.21.15.134:0/1418350955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90e0069930 msgr2=0x7f90e014ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:14.340 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dffff700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90e0069930 0x7f90e014ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.340 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dffff700 1 -- 172.21.15.134:0/1418350955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc009550 con 0x7f90e00ce7d0 2022-11-27T16:17:14.340 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dffff700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 0x7f90e006db10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90cc000c00 tx=0x7f90cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:14.340 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dd7fa700 1 -- 172.21.15.134:0/1418350955 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc004480 con 0x7f90e00ce7d0 2022-11-27T16:17:14.341 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.325+0000 7f90dd7fa700 1 -- 172.21.15.134:0/1418350955 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc0045e0 con 0x7f90e00ce7d0 2022-11-27T16:17:14.341 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.326+0000 7f90e720b700 1 -- 172.21.15.134:0/1418350955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e014f710 con 0x7f90e00ce7d0 2022-11-27T16:17:14.341 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.326+0000 7f90dd7fa700 1 -- 172.21.15.134:0/1418350955 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0038e0 con 0x7f90e00ce7d0 2022-11-27T16:17:14.341 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.326+0000 7f90e720b700 1 -- 172.21.15.134:0/1418350955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e014fcd0 con 0x7f90e00ce7d0 2022-11-27T16:17:14.342 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.326+0000 7f90e720b700 1 -- 172.21.15.134:0/1418350955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e00aee70 con 0x7f90e00ce7d0 2022-11-27T16:17:14.342 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.327+0000 7f90dd7fa700 1 -- 172.21.15.134:0/1418350955 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f90cc00f050 con 0x7f90e00ce7d0 2022-11-27T16:17:14.342 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.327+0000 7f90dd7fa700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f90c8066590 0x7f90c8068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.342 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.327+0000 7f90dd7fa700 1 -- 172.21.15.134:0/1418350955 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f90cc084ab0 con 0x7f90e00ce7d0 2022-11-27T16:17:14.342 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.327+0000 7f90e4fa7700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f90c8066590 0x7f90c8068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.343 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.328+0000 7f90e4fa7700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f90c8066590 0x7f90c8068a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90d4006f10 tx=0x7f90d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:14.343 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.332+0000 7f90dd7fa700 1 -- 172.21.15.134:0/1418350955 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f90cc052670 con 0x7f90e00ce7d0 2022-11-27T16:17:14.514 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.512+0000 7f90e720b700 1 -- 172.21.15.134:0/1418350955 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90e0073030 con 0x7f90e00ce7d0 2022-11-27T16:17:14.515 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.513+0000 7f90dd7fa700 1 -- 172.21.15.134:0/1418350955 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12269 (secure 0 0 0) 0x7f90cc015090 con 0x7f90e00ce7d0 2022-11-27T16:17:14.516 INFO:teuthology.misc.health.smithi134.stdout: 2022-11-27T16:17:14.516 INFO:teuthology.misc.health.smithi134.stdout:{"epoch":17,"fsid":"34bf4849-eb1e-4259-a7f0-0be8a82c33c9","created":"2022-11-27T16:16:56.408900+0000","modified":"2022-11-27T16:17:12.197367+0000","last_up_change":"2022-11-27T16:17:07.774037+0000","last_in_change":"2022-11-27T16:16:58.964080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T16:17:09.769201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"f44064b8-6282-45ff-94ab-f3a2eb231dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":101352},{"type":"v1","addr":"172.21.15.134:6809","nonce":101352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":101352},{"type":"v1","addr":"172.21.15.134:6811","nonce":101352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":101352},{"type":"v1","addr":"172.21.15.134:6815","nonce":101352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":101352},{"type":"v1","addr":"172.21.15.134:6813","nonce":101352}]},"public_addr":"172.21.15.134:6809/101352","cluster_addr":"172.21.15.134:6811/101352","heartbeat_back_addr":"172.21.15.134:6815/101352","heartbeat_front_addr":"172.21.15.134:6813/101352","state":["exists","up"]},{"osd":1,"uuid":"af1b8ac9-0335-4550-aaf8-9f15545625e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":101351},{"type":"v1","addr":"172.21.15.134:6801","nonce":101351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":101351},{"type":"v1","addr":"172.21.15.134:6803","nonce":101351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":101351},{"type":"v1","addr":"172.21.15.134:6807","nonce":101351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":101351},{"type":"v1","addr":"172.21.15.134:6805","nonce":101351}]},"public_addr":"172.21.15.134:6801/101351","cluster_addr":"172.21.15.134:6803/101351","heartbeat_back_addr":"172.21.15.134:6807/101351","heartbeat_front_addr":"172.21.15.134:6805/101351","state":["exists","up"]},{"osd":2,"uuid":"5bbdd443-3ec5-40fb-86af-3e5a6ab1606a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":101353},{"type":"v1","addr":"172.21.15.134:6817","nonce":101353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":101353},{"type":"v1","addr":"172.21.15.134:6819","nonce":101353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":101353},{"type":"v1","addr":"172.21.15.134:6823","nonce":101353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":101353},{"type":"v1","addr":"172.21.15.134:6821","nonce":101353}]},"public_addr":"172.21.15.134:6817/101353","cluster_addr":"172.21.15.134:6819/101353","heartbeat_back_addr":"172.21.15.134:6823/101353","heartbeat_front_addr":"172.21.15.134:6821/101353","state":["exists","up"]},{"osd":3,"uuid":"795fd2fb-0e2e-4419-9b54-ea28ba0ee835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":101371},{"type":"v1","addr":"172.21.15.134:6825","nonce":101371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":101371},{"type":"v1","addr":"172.21.15.134:6827","nonce":101371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":101371},{"type":"v1","addr":"172.21.15.134:6831","nonce":101371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":101371},{"type":"v1","addr":"172.21.15.134:6829","nonce":101371}]},"public_addr":"172.21.15.134:6825/101371","cluster_addr":"172.21.15.134:6827/101371","heartbeat_back_addr":"172.21.15.134:6831/101371","heartbeat_front_addr":"172.21.15.134:6829/101371","state":["exists","up"]},{"osd":4,"uuid":"36005e5e-73a9-4d10-aba3-480165df1594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":101536},{"type":"v1","addr":"172.21.15.191:6817","nonce":101536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":101536},{"type":"v1","addr":"172.21.15.191:6819","nonce":101536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":101536},{"type":"v1","addr":"172.21.15.191:6823","nonce":101536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":101536},{"type":"v1","addr":"172.21.15.191:6821","nonce":101536}]},"public_addr":"172.21.15.191:6817/101536","cluster_addr":"172.21.15.191:6819/101536","heartbeat_back_addr":"172.21.15.191:6823/101536","heartbeat_front_addr":"172.21.15.191:6821/101536","state":["exists","up"]},{"osd":5,"uuid":"6f5b5e55-e69b-4097-a47c-cd7fc0681dbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":101534},{"type":"v1","addr":"172.21.15.191:6801","nonce":101534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":101534},{"type":"v1","addr":"172.21.15.191:6803","nonce":101534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":101534},{"type":"v1","addr":"172.21.15.191:6807","nonce":101534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":101534},{"type":"v1","addr":"172.21.15.191:6805","nonce":101534}]},"public_addr":"172.21.15.191:6801/101534","cluster_addr":"172.21.15.191:6803/101534","heartbeat_back_addr":"172.21.15.191:6807/101534","heartbeat_front_addr":"172.21.15.191:6805/101534","state":["exists","up"]},{"osd":6,"uuid":"074077c9-fc46-4f79-8368-5a96eba6da06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":101545},{"type":"v1","addr":"172.21.15.191:6825","nonce":101545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":101545},{"type":"v1","addr":"172.21.15.191:6827","nonce":101545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":101545},{"type":"v1","addr":"172.21.15.191:6831","nonce":101545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":101545},{"type":"v1","addr":"172.21.15.191:6829","nonce":101545}]},"public_addr":"172.21.15.191:6825/101545","cluster_addr":"172.21.15.191:6827/101545","heartbeat_back_addr":"172.21.15.191:6831/101545","heartbeat_front_addr":"172.21.15.191:6829/101545","state":["exists","up"]},{"osd":7,"uuid":"112889bf-b637-4500-a30d-03e23e8a3267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":101535},{"type":"v1","addr":"172.21.15.191:6809","nonce":101535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":101535},{"type":"v1","addr":"172.21.15.191:6811","nonce":101535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":101535},{"type":"v1","addr":"172.21.15.191:6815","nonce":101535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":101535},{"type":"v1","addr":"172.21.15.191:6813","nonce":101535}]},"public_addr":"172.21.15.191:6809/101535","cluster_addr":"172.21.15.191:6811/101535","heartbeat_back_addr":"172.21.15.191:6815/101535","heartbeat_front_addr":"172.21.15.191:6813/101535","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.141813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:03.885025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.127132+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.180968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.193639+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.201380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.195434+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.198016+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:17:14.517 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 -- 172.21.15.134:0/1418350955 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f90c8066590 msgr2=0x7f90c8068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:14.518 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f90c8066590 0x7f90c8068a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90d4006f10 tx=0x7f90d4008040 comp rx=0 tx=0).stop 2022-11-27T16:17:14.518 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 -- 172.21.15.134:0/1418350955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 msgr2=0x7f90e006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:14.518 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 0x7f90e006db10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90cc000c00 tx=0x7f90cc004750 comp rx=0 tx=0).stop 2022-11-27T16:17:14.518 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 -- 172.21.15.134:0/1418350955 shutdown_connections 2022-11-27T16:17:14.519 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90e015b9b0 0x7f90e0149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.519 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f90c8066590 0x7f90c8068a40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.519 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90e0069930 0x7f90e014ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.519 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 --2- 172.21.15.134:0/1418350955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e00ce7d0 0x7f90e006db10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.520 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 -- 172.21.15.134:0/1418350955 >> 172.21.15.134:0/1418350955 conn(0x7f90e00b0aa0 msgr2=0x7f90e00c94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:14.520 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 -- 172.21.15.134:0/1418350955 shutdown_connections 2022-11-27T16:17:14.520 INFO:teuthology.misc.health.smithi134.stderr:2022-11-27T16:17:14.515+0000 7f90e720b700 1 -- 172.21.15.134:0/1418350955 wait complete. 2022-11-27T16:17:14.528 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-11-27T16:17:14.529 INFO:tasks.ceph:Creating RBD pool 2022-11-27T16:17:14.529 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd pool create rbd 8 2022-11-27T16:17:14.657 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.646+0000 7f14f6266700 1 Processor -- start 2022-11-27T16:17:14.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.646+0000 7f14f6266700 1 -- start start 2022-11-27T16:17:14.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14f6266700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f01598c0 0x7f14f00d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14f6266700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f14f00b34d0 0x7f14f006d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14f6266700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f00b1ca0 con 0x7f14f00d0ae0 2022-11-27T16:17:14.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14f6266700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f00cc830 con 0x7f14f00b34d0 2022-11-27T16:17:14.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14f6266700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00cc270 con 0x7f14f01598c0 2022-11-27T16:17:14.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14effff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f01598c0 0x7f14f00d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14effff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f01598c0 0x7f14f00d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.134:41744/0 (socket says 172.21.15.134:41744) 2022-11-27T16:17:14.660 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14effff700 1 -- 172.21.15.134:0/1688090009 learned_addr learned my addr 172.21.15.134:0/1688090009 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:14.660 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14f4803700 1 --2- 172.21.15.134:0/1688090009 >> v2:172.21.15.191:3300/0 conn(0x7f14f00b34d0 0x7f14f006d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.660 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14effff700 1 -- 172.21.15.134:0/1688090009 >> v2:172.21.15.191:3300/0 conn(0x7f14f00b34d0 msgr2=0x7f14f006d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:14.660 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14effff700 1 --2- 172.21.15.134:0/1688090009 >> v2:172.21.15.191:3300/0 conn(0x7f14f00b34d0 0x7f14f006d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.661 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14effff700 1 -- 172.21.15.134:0/1688090009 >> v1:172.21.15.191:6789/0 conn(0x7f14f00d0ae0 legacy=0x7f14f00b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:14.661 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14eeffd700 1 -- 172.21.15.134:0/1688090009 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743845367 0 0) 0x7f14f00b1ca0 con 0x7f14f00d0ae0 2022-11-27T16:17:14.661 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.647+0000 7f14effff700 1 -- 172.21.15.134:0/1688090009 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f00cc550 con 0x7f14f01598c0 2022-11-27T16:17:14.661 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14effff700 1 --2- 172.21.15.134:0/1688090009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f01598c0 0x7f14f00d05a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f14e40098f0 tx=0x7f14e4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=142836ec67bb3c31 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:14.661 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14eeffd700 1 -- 172.21.15.134:0/1688090009 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4011830 con 0x7f14f01598c0 2022-11-27T16:17:14.662 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14eeffd700 1 -- 172.21.15.134:0/1688090009 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e4011990 con 0x7f14f01598c0 2022-11-27T16:17:14.662 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14eeffd700 1 -- 172.21.15.134:0/1688090009 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4011d20 con 0x7f14f01598c0 2022-11-27T16:17:14.662 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14f6266700 1 -- 172.21.15.134:0/1688090009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f01598c0 msgr2=0x7f14f00d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:14.662 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14f6266700 1 --2- 172.21.15.134:0/1688090009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f01598c0 0x7f14f00d05a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f14e40098f0 tx=0x7f14e4009c00 comp rx=0 tx=0).stop 2022-11-27T16:17:14.663 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14f6266700 1 -- 172.21.15.134:0/1688090009 shutdown_connections 2022-11-27T16:17:14.663 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14f6266700 1 --2- 172.21.15.134:0/1688090009 >> v2:172.21.15.191:3300/0 conn(0x7f14f00b34d0 0x7f14f006d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.663 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14f6266700 1 --2- 172.21.15.134:0/1688090009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f01598c0 0x7f14f00d05a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.663 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14f6266700 1 -- 172.21.15.134:0/1688090009 >> 172.21.15.134:0/1688090009 conn(0x7f14f0071060 msgr2=0x7f14f0071430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:14.664 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14f6266700 1 -- 172.21.15.134:0/1688090009 shutdown_connections 2022-11-27T16:17:14.664 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.648+0000 7f14f6266700 1 -- 172.21.15.134:0/1688090009 wait complete. 2022-11-27T16:17:14.664 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.650+0000 7f14f6266700 1 Processor -- start 2022-11-27T16:17:14.664 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.650+0000 7f14f6266700 1 -- start start 2022-11-27T16:17:14.664 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f6266700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f00b34d0 0x7f14f00c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.665 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f6266700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f00d0ae0 0x7f14f015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.665 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f6266700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14f01598c0 0x7f14f0153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.665 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f6266700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00c8330 con 0x7f14f01598c0 2022-11-27T16:17:14.665 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f6266700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f015cc10 con 0x7f14f00b34d0 2022-11-27T16:17:14.666 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f6266700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00c76b0 con 0x7f14f00d0ae0 2022-11-27T16:17:14.666 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f4803700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14f01598c0 0x7f14f0153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.666 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f4803700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14f01598c0 0x7f14f0153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37970/0 (socket says 172.21.15.134:37970) 2022-11-27T16:17:14.666 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f4803700 1 -- 172.21.15.134:0/1788860327 learned_addr learned my addr 172.21.15.134:0/1788860327 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:14.666 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14effff700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f00b34d0 0x7f14f00c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.667 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14ef7fe700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f00d0ae0 0x7f14f015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.667 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f4803700 1 -- 172.21.15.134:0/1788860327 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f00d0ae0 msgr2=0x7f14f015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:14.667 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f4803700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f00d0ae0 0x7f14f015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.668 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f4803700 1 -- 172.21.15.134:0/1788860327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f00b34d0 msgr2=0x7f14f00c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:14.668 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f4803700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f00b34d0 0x7f14f00c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:14.668 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14f4803700 1 -- 172.21.15.134:0/1788860327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e4009550 con 0x7f14f01598c0 2022-11-27T16:17:14.668 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14effff700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f00b34d0 0x7f14f00c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:17:14.669 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.651+0000 7f14ef7fe700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f00d0ae0 0x7f14f015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:17:14.669 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.652+0000 7f14f4803700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14f01598c0 0x7f14f0153280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14e800ef90 tx=0x7f14e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:14.669 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.652+0000 7f14ecff9700 1 -- 172.21.15.134:0/1788860327 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8009b20 con 0x7f14f01598c0 2022-11-27T16:17:14.669 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.652+0000 7f14ecff9700 1 -- 172.21.15.134:0/1788860327 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e800f040 con 0x7f14f01598c0 2022-11-27T16:17:14.669 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.652+0000 7f14ecff9700 1 -- 172.21.15.134:0/1788860327 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8009e50 con 0x7f14f01598c0 2022-11-27T16:17:14.670 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.652+0000 7f14f6266700 1 -- 172.21.15.134:0/1788860327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f00c7990 con 0x7f14f01598c0 2022-11-27T16:17:14.670 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.652+0000 7f14f6266700 1 -- 172.21.15.134:0/1788860327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f00c7f50 con 0x7f14f01598c0 2022-11-27T16:17:14.670 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.653+0000 7f14f6266700 1 -- 172.21.15.134:0/1788860327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f0073030 con 0x7f14f01598c0 2022-11-27T16:17:14.670 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.653+0000 7f14ecff9700 1 -- 172.21.15.134:0/1788860327 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f14e8014480 con 0x7f14f01598c0 2022-11-27T16:17:14.671 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.654+0000 7f14ecff9700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f14d0066590 0x7f14d0068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:14.671 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.654+0000 7f14effff700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f14d0066590 0x7f14d0068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:14.671 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.654+0000 7f14ecff9700 1 -- 172.21.15.134:0/1788860327 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f14e8084010 con 0x7f14f01598c0 2022-11-27T16:17:14.671 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.654+0000 7f14effff700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f14d0066590 0x7f14d0068a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009c40 tx=0x7f14e4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:14.672 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.658+0000 7f14ecff9700 1 -- 172.21.15.134:0/1788860327 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f14e8051b70 con 0x7f14f01598c0 2022-11-27T16:17:14.839 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:14.837+0000 7f14f6266700 1 -- 172.21.15.134:0/1788860327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f14f00ca940 con 0x7f14f01598c0 2022-11-27T16:17:15.806 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.804+0000 7f14ecff9700 1 -- 172.21.15.134:0/1788860327 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7f14e8055190 con 0x7f14f01598c0 2022-11-27T16:17:15.810 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.807+0000 7f14f6266700 1 -- 172.21.15.134:0/1788860327 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f14d0066590 msgr2=0x7f14d0068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:15.810 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.807+0000 7f14f6266700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f14d0066590 0x7f14d0068a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009c40 tx=0x7f14e4008030 comp rx=0 tx=0).stop 2022-11-27T16:17:15.810 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.807+0000 7f14f6266700 1 -- 172.21.15.134:0/1788860327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14f01598c0 msgr2=0x7f14f0153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:15.811 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.807+0000 7f14f6266700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14f01598c0 0x7f14f0153280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14e800ef90 tx=0x7f14e800c3a0 comp rx=0 tx=0).stop 2022-11-27T16:17:15.811 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.808+0000 7f14f6266700 1 -- 172.21.15.134:0/1788860327 shutdown_connections 2022-11-27T16:17:15.811 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.808+0000 7f14f6266700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14f01598c0 0x7f14f0153280 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:15.812 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.808+0000 7f14f6266700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f14d0066590 0x7f14d0068a40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:15.812 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.808+0000 7f14f6266700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14f00b34d0 0x7f14f00c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:15.812 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.808+0000 7f14f6266700 1 --2- 172.21.15.134:0/1788860327 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14f00d0ae0 0x7f14f015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:15.812 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.808+0000 7f14f6266700 1 -- 172.21.15.134:0/1788860327 >> 172.21.15.134:0/1788860327 conn(0x7f14f0071060 msgr2=0x7f14f00cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:15.813 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.808+0000 7f14f6266700 1 -- 172.21.15.134:0/1788860327 shutdown_connections 2022-11-27T16:17:15.813 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.808+0000 7f14f6266700 1 -- 172.21.15.134:0/1788860327 wait complete. 2022-11-27T16:17:15.813 INFO:teuthology.orchestra.run.smithi134.stderr:pool 'rbd' created 2022-11-27T16:17:15.824 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-11-27T16:17:15.952 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.950+0000 7fc4c1422700 1 Processor -- start 2022-11-27T16:17:15.953 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.951+0000 7fc4c1422700 1 -- start start 2022-11-27T16:17:15.960 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.951+0000 7fc4c1422700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0ccf30 0x7fc4bc15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:15.960 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.951+0000 7fc4c1422700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4bc15d110 con 0x7fc4bc15c920 2022-11-27T16:17:15.961 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.951+0000 7fc4c1422700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4bc15f840 con 0x7fc4bc159670 2022-11-27T16:17:15.961 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.951+0000 7fc4c1422700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc1601e0 con 0x7fc4bc0ccf30 2022-11-27T16:17:15.961 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.951+0000 7fc4bb7fe700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc4bc15c920 0x7fc4bc0d01c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56046/0 (socket says 172.21.15.134:56046) 2022-11-27T16:17:15.961 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.951+0000 7fc4bb7fe700 1 -- 172.21.15.134:0/3243548605 learned_addr learned my addr 172.21.15.134:0/3243548605 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:17:15.962 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4ba7fc700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0ccf30 0x7fc4bc15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:15.962 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317641775 0 0) 0x7fc4bc15d110 con 0x7fc4bc15c920 2022-11-27T16:17:15.962 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a80035b0 con 0x7fc4bc15c920 2022-11-27T16:17:15.962 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3814532251 0 0) 0x7fc4a80035b0 con 0x7fc4bc15c920 2022-11-27T16:17:15.962 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0ccf30 msgr2=0x7fc4bc15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:15.963 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0ccf30 0x7fc4bc15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:15.963 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 >> v1:172.21.15.191:6789/0 conn(0x7fc4bc159670 legacy=0x7fc4bc0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:15.963 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc15fb20 con 0x7fc4bc15c920 2022-11-27T16:17:15.963 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2726658286 0 0) 0x7fc4b40019f0 con 0x7fc4bc15c920 2022-11-27T16:17:15.964 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 >> v1:172.21.15.134:6789/0 conn(0x7fc4bc15c920 legacy=0x7fc4bc0d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:15.964 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a8003a30 0x7fc4a8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:15.964 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a8004850 0x7fc4a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:15.965 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0ccf30 0x7fc4a80052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:15.965 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc1601e0 con 0x7fc4a8004850 2022-11-27T16:17:15.965 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a800bc00 con 0x7fc4a8003a30 2022-11-27T16:17:15.966 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a800bdc0 con 0x7fc4bc0ccf30 2022-11-27T16:17:15.966 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4ba7fc700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a8004850 0x7fc4a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:15.966 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4baffd700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a8003a30 0x7fc4a8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:15.966 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4bb7fe700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0ccf30 0x7fc4a80052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:15.967 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4ba7fc700 1 -- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0ccf30 msgr2=0x7fc4a80052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:15.967 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4ba7fc700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0ccf30 0x7fc4a80052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:15.967 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4ba7fc700 1 -- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a8003a30 msgr2=0x7fc4a8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:15.968 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4ba7fc700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a8003a30 0x7fc4a8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:15.968 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.952+0000 7fc4ba7fc700 1 -- 172.21.15.134:0/3243548605 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a80035b0 con 0x7fc4a8004850 2022-11-27T16:17:15.968 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4ba7fc700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a8004850 0x7fc4a8004cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0cd320 tx=0x7fc4a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1127c583cf91f768 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:15.968 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00e6c0 con 0x7fc4a8004850 2022-11-27T16:17:15.969 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac00e820 con 0x7fc4a8004850 2022-11-27T16:17:15.969 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4b9ffb700 1 -- 172.21.15.134:0/3243548605 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00eb90 con 0x7fc4a8004850 2022-11-27T16:17:15.969 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4c1422700 1 -- 172.21.15.134:0/3243548605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a8004850 msgr2=0x7fc4a8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:15.969 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4c1422700 1 --2- 172.21.15.134:0/3243548605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a8004850 0x7fc4a8004cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0cd320 tx=0x7fc4a8005880 comp rx=0 tx=0).stop 2022-11-27T16:17:15.970 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4bb7fe700 1 -- 172.21.15.134:0/3243548605 reap_dead start 2022-11-27T16:17:15.970 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4c1422700 1 -- 172.21.15.134:0/3243548605 shutdown_connections 2022-11-27T16:17:15.970 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4c1422700 1 -- 172.21.15.134:0/3243548605 >> 172.21.15.134:0/3243548605 conn(0x7fc4bc069df0 msgr2=0x7fc4bc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:15.970 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4c1422700 1 -- 172.21.15.134:0/3243548605 shutdown_connections 2022-11-27T16:17:15.970 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.953+0000 7fc4c1422700 1 -- 172.21.15.134:0/3243548605 wait complete. 2022-11-27T16:17:15.971 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4c1422700 1 Processor -- start 2022-11-27T16:17:15.971 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4c1422700 1 -- start start 2022-11-27T16:17:15.971 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4c1422700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a8004850 0x7fc4bc14d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:15.972 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4c1422700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0b7570 0x7fc4bc0b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:15.972 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4c1422700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc15d880 0x7fc4bc15dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:15.972 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4c1422700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc15e230 con 0x7fc4bc15d880 2022-11-27T16:17:15.972 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4c1422700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0c6840 con 0x7fc4a8004850 2022-11-27T16:17:15.972 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4c1422700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0b5e60 con 0x7fc4bc0b7570 2022-11-27T16:17:15.973 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4bb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc15d880 0x7fc4bc15dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:15.973 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4bb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc15d880 0x7fc4bc15dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37986/0 (socket says 172.21.15.134:37986) 2022-11-27T16:17:15.973 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.955+0000 7fc4bb7fe700 1 -- 172.21.15.134:0/911858115 learned_addr learned my addr 172.21.15.134:0/911858115 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:15.973 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4ba7fc700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0b7570 0x7fc4bc0b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:15.974 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4baffd700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a8004850 0x7fc4bc14d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:15.974 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4bb7fe700 1 -- 172.21.15.134:0/911858115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0b7570 msgr2=0x7fc4bc0b79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:15.974 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4bb7fe700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0b7570 0x7fc4bc0b79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:15.974 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4bb7fe700 1 -- 172.21.15.134:0/911858115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a8004850 msgr2=0x7fc4bc14d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:15.975 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4bb7fe700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a8004850 0x7fc4bc14d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:15.975 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4bb7fe700 1 -- 172.21.15.134:0/911858115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc0b5000 con 0x7fc4bc15d880 2022-11-27T16:17:15.975 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4bb7fe700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc15d880 0x7fc4bc15dcf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0cdfc0 tx=0x7fc4b400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:15.975 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4a37fe700 1 -- 172.21.15.134:0/911858115 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400ac30 con 0x7fc4bc15d880 2022-11-27T16:17:15.975 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4a37fe700 1 -- 172.21.15.134:0/911858115 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b400ad90 con 0x7fc4bc15d880 2022-11-27T16:17:15.976 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4c1422700 1 -- 172.21.15.134:0/911858115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc0b52e0 con 0x7fc4bc15d880 2022-11-27T16:17:15.976 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4a37fe700 1 -- 172.21.15.134:0/911858115 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4013630 con 0x7fc4bc15d880 2022-11-27T16:17:15.976 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.956+0000 7fc4c1422700 1 -- 172.21.15.134:0/911858115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc0b58a0 con 0x7fc4bc15d880 2022-11-27T16:17:15.976 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.957+0000 7fc4c1422700 1 -- 172.21.15.134:0/911858115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc0c69d0 con 0x7fc4bc15d880 2022-11-27T16:17:15.977 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.957+0000 7fc4a37fe700 1 -- 172.21.15.134:0/911858115 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fc4b4013790 con 0x7fc4bc15d880 2022-11-27T16:17:15.977 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.958+0000 7fc4a37fe700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc49c066590 0x7fc49c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:15.977 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.958+0000 7fc4baffd700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc49c066590 0x7fc49c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:15.977 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.958+0000 7fc4a37fe700 1 -- 172.21.15.134:0/911858115 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fc4b4083d70 con 0x7fc4bc15d880 2022-11-27T16:17:15.978 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.958+0000 7fc4baffd700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc49c066590 0x7fc49c068a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0007fd0 tx=0x7fc4b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:15.979 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:15.962+0000 7fc4a37fe700 1 -- 172.21.15.134:0/911858115 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fc4bc0c69d0 con 0x7fc4bc15d880 2022-11-27T16:17:16.142 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.140+0000 7fc4c1422700 1 -- 172.21.15.134:0/911858115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc4bc0c69d0 con 0x7fc4bc15d880 2022-11-27T16:17:16.808 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.806+0000 7fc4a37fe700 1 -- 172.21.15.134:0/911858115 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc4bc0c69d0 con 0x7fc4bc15d880 2022-11-27T16:17:16.812 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 -- 172.21.15.134:0/911858115 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc49c066590 msgr2=0x7fc49c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:16.812 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc49c066590 0x7fc49c068a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0007fd0 tx=0x7fc4b0009040 comp rx=0 tx=0).stop 2022-11-27T16:17:16.813 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 -- 172.21.15.134:0/911858115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc15d880 msgr2=0x7fc4bc15dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:16.813 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc15d880 0x7fc4bc15dcf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0cdfc0 tx=0x7fc4b400a5f0 comp rx=0 tx=0).stop 2022-11-27T16:17:16.813 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 -- 172.21.15.134:0/911858115 shutdown_connections 2022-11-27T16:17:16.813 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4bc15d880 0x7fc4bc15dcf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:16.814 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc49c066590 0x7fc49c068a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:16.814 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4a8004850 0x7fc4bc14d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:16.814 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 --2- 172.21.15.134:0/911858115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4bc0b7570 0x7fc4bc0b79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:16.814 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 -- 172.21.15.134:0/911858115 >> 172.21.15.134:0/911858115 conn(0x7fc4bc069df0 msgr2=0x7fc4bc159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:16.814 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 -- 172.21.15.134:0/911858115 shutdown_connections 2022-11-27T16:17:16.815 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.810+0000 7fc4c1422700 1 -- 172.21.15.134:0/911858115 wait complete. 2022-11-27T16:17:16.815 INFO:teuthology.orchestra.run.smithi134.stderr:enabled application 'rbd' on pool 'rbd' 2022-11-27T16:17:16.829 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-11-27T16:17:16.829 INFO:tasks.ceph.mds.a:Restarting daemon 2022-11-27T16:17:16.830 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-11-27T16:17:16.832 INFO:tasks.ceph.mds.a:Started 2022-11-27T16:17:16.833 INFO:tasks.ceph.mds.c:Restarting daemon 2022-11-27T16:17:16.833 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-11-27T16:17:16.835 INFO:tasks.ceph.mds.c:Started 2022-11-27T16:17:16.836 INFO:tasks.ceph.mds.b:Restarting daemon 2022-11-27T16:17:16.836 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-11-27T16:17:16.839 INFO:tasks.ceph.mds.b:Started 2022-11-27T16:17:16.840 INFO:tasks.ceph.mds.d:Restarting daemon 2022-11-27T16:17:16.840 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2022-11-27T16:17:16.842 INFO:tasks.ceph.mds.d:Started 2022-11-27T16:17:16.843 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-11-27T16:17:16.844 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:17:16.917 INFO:tasks.ceph.mds.a.smithi134.stdout:starting mds.a at 2022-11-27T16:17:16.918 INFO:tasks.ceph.mds.c.smithi134.stdout:starting mds.c at 2022-11-27T16:17:16.930 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2022-11-27T16:17:16.931 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2022-11-27T16:17:16.977 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.976+0000 7f02f8eb6700 1 Processor -- start 2022-11-27T16:17:16.978 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.976+0000 7f02f8eb6700 1 -- start start 2022-11-27T16:17:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.976+0000 7f02f8eb6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f4158830 0x7f02f40cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f8eb6700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f415c830 con 0x7f02f40b0aa0 2022-11-27T16:17:16.987 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f8eb6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f41596f0 con 0x7f02f415ccc0 2022-11-27T16:17:16.987 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f8eb6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f41599d0 con 0x7f02f4158830 2022-11-27T16:17:16.987 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f2d9d700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f02f40b0aa0 0x7f02f406ff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56092/0 (socket says 172.21.15.134:56092) 2022-11-27T16:17:16.987 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f2d9d700 1 -- 172.21.15.134:0/1477361359 learned_addr learned my addr 172.21.15.134:0/1477361359 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:17:16.988 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f259c700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f4158830 0x7f02f40cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:16.988 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274909290 0 0) 0x7f02f415c830 con 0x7f02f40b0aa0 2022-11-27T16:17:16.988 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d80035b0 con 0x7f02f40b0aa0 2022-11-27T16:17:16.988 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520116959 0 0) 0x7f02f41596f0 con 0x7f02f415ccc0 2022-11-27T16:17:16.989 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02f415c830 con 0x7f02f415ccc0 2022-11-27T16:17:16.989 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 546902476 0 0) 0x7f02d80035b0 con 0x7f02f40b0aa0 2022-11-27T16:17:16.989 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f4158830 msgr2=0x7f02f40cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:16.989 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f4158830 0x7f02f40cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:16.989 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 >> v1:172.21.15.191:6789/0 conn(0x7f02f415ccc0 legacy=0x7f02f4158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:16.990 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f415c4c0 con 0x7f02f40b0aa0 2022-11-27T16:17:16.990 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2726658286 0 0) 0x7f02ec0019f0 con 0x7f02f40b0aa0 2022-11-27T16:17:16.990 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 >> v1:172.21.15.134:6789/0 conn(0x7f02f40b0aa0 legacy=0x7f02f406ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:16.990 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8003bc0 0x7f02d8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:16.990 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02d8004850 0x7f02d8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:16.991 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f4158830 0x7f02d8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:16.991 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f41599d0 con 0x7f02d8003bc0 2022-11-27T16:17:16.991 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d800bac0 con 0x7f02d8004850 2022-11-27T16:17:16.991 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d800bc30 con 0x7f02f4158830 2022-11-27T16:17:16.992 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f259c700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8003bc0 0x7f02d8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:16.992 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f1d9b700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02d8004850 0x7f02d8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:16.992 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.977+0000 7f02f2d9d700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f4158830 0x7f02d8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:16.993 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f259c700 1 -- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f4158830 msgr2=0x7f02d8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:16.993 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f259c700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f4158830 0x7f02d8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:16.993 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f259c700 1 -- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02d8004850 msgr2=0x7f02d8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f259c700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02d8004850 0x7f02d8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f259c700 1 -- 172.21.15.134:0/1477361359 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80035b0 con 0x7f02d8003bc0 2022-11-27T16:17:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f259c700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8003bc0 0x7f02d8003fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009b60 tx=0x7f02d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf5a685d33a3b9 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400e6c0 con 0x7f02d8003bc0 2022-11-27T16:17:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e400e820 con 0x7f02d8003bc0 2022-11-27T16:17:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f159a700 1 -- 172.21.15.134:0/1477361359 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400eb90 con 0x7f02d8003bc0 2022-11-27T16:17:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f8eb6700 1 -- 172.21.15.134:0/1477361359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8003bc0 msgr2=0x7f02d8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f8eb6700 1 --2- 172.21.15.134:0/1477361359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02d8003bc0 0x7f02d8003fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009b60 tx=0x7f02d8005880 comp rx=0 tx=0).stop 2022-11-27T16:17:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f2d9d700 1 -- 172.21.15.134:0/1477361359 reap_dead start 2022-11-27T16:17:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f8eb6700 1 -- 172.21.15.134:0/1477361359 shutdown_connections 2022-11-27T16:17:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.978+0000 7f02f8eb6700 1 -- 172.21.15.134:0/1477361359 >> 172.21.15.134:0/1477361359 conn(0x7f02f406a500 msgr2=0x7f02f406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.979+0000 7f02f8eb6700 1 -- 172.21.15.134:0/1477361359 shutdown_connections 2022-11-27T16:17:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.979+0000 7f02f8eb6700 1 -- 172.21.15.134:0/1477361359 wait complete. 2022-11-27T16:17:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.980+0000 7f02f8eb6700 1 Processor -- start 2022-11-27T16:17:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.980+0000 7f02f8eb6700 1 -- start start 2022-11-27T16:17:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f8eb6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02d8003bc0 0x7f02f40b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f8eb6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f40aff40 0x7f02f4153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f8eb6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f41563c0 0x7f02f40b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f8eb6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f4152d70 con 0x7f02f41563c0 2022-11-27T16:17:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f8eb6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f40b0500 con 0x7f02d8003bc0 2022-11-27T16:17:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f8eb6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f4152720 con 0x7f02f40aff40 2022-11-27T16:17:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f2d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f41563c0 0x7f02f40b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f259c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02d8003bc0 0x7f02f40b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f1d9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f40aff40 0x7f02f4153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f2d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f41563c0 0x7f02f40b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38032/0 (socket says 172.21.15.134:38032) 2022-11-27T16:17:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f259c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02d8003bc0 0x7f02f40b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.134:54424/0 (socket says 172.21.15.134:54424) 2022-11-27T16:17:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f2d9d700 1 -- 172.21.15.134:0/3644955231 learned_addr learned my addr 172.21.15.134:0/3644955231 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f2d9d700 1 -- 172.21.15.134:0/3644955231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f40aff40 msgr2=0x7f02f4153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f2d9d700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f40aff40 0x7f02f4153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f2d9d700 1 -- 172.21.15.134:0/3644955231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02d8003bc0 msgr2=0x7f02f40b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f2d9d700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02d8003bc0 0x7f02f40b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f2d9d700 1 -- 172.21.15.134:0/3644955231 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f4152a00 con 0x7f02f41563c0 2022-11-27T16:17:17.008 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f2d9d700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f41563c0 0x7f02f40b3850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02f40cddb0 tx=0x7f02ec00a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.008 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02e2ffd700 1 -- 172.21.15.134:0/3644955231 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec00acc0 con 0x7f02f41563c0 2022-11-27T16:17:17.008 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02e2ffd700 1 -- 172.21.15.134:0/3644955231 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ec00ae40 con 0x7f02f41563c0 2022-11-27T16:17:17.009 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02e2ffd700 1 -- 172.21.15.134:0/3644955231 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec013630 con 0x7f02f41563c0 2022-11-27T16:17:17.009 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f8eb6700 1 -- 172.21.15.134:0/3644955231 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f40b2350 con 0x7f02f41563c0 2022-11-27T16:17:17.010 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.981+0000 7f02f8eb6700 1 -- 172.21.15.134:0/3644955231 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f4153740 con 0x7f02f41563c0 2022-11-27T16:17:17.010 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.982+0000 7f02f8eb6700 1 -- 172.21.15.134:0/3644955231 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f4153930 con 0x7f02f41563c0 2022-11-27T16:17:17.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.983+0000 7f02e2ffd700 1 -- 172.21.15.134:0/3644955231 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f02ec023070 con 0x7f02f41563c0 2022-11-27T16:17:17.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.983+0000 7f02e2ffd700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f02d0066580 0x7f02d0068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.983+0000 7f02e2ffd700 1 -- 172.21.15.134:0/3644955231 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f02ec083d70 con 0x7f02f41563c0 2022-11-27T16:17:17.012 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.983+0000 7f02f259c700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f02d0066580 0x7f02d0068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.013 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.984+0000 7f02f259c700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f02d0066580 0x7f02d0068a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f02e40090b0 tx=0x7f02e401f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.014 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:16.988+0000 7f02e2ffd700 1 -- 172.21.15.134:0/3644955231 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f02ec0517d0 con 0x7f02f41563c0 2022-11-27T16:17:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.167+0000 7f02f8eb6700 1 -- 172.21.15.134:0/3644955231 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02f4073030 con 0x7f02f41563c0 2022-11-27T16:17:17.170 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.167+0000 7f02e2ffd700 1 -- 172.21.15.134:0/3644955231 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13802 (secure 0 0 0) 0x7f02ec017020 con 0x7f02f41563c0 2022-11-27T16:17:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:17:17.171 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":19,"fsid":"34bf4849-eb1e-4259-a7f0-0be8a82c33c9","created":"2022-11-27T16:16:56.408900+0000","modified":"2022-11-27T16:17:16.803801+0000","last_up_change":"2022-11-27T16:17:07.774037+0000","last_in_change":"2022-11-27T16:16:58.964080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T16:17:09.769201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T16:17:14.856029+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f44064b8-6282-45ff-94ab-f3a2eb231dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":101352},{"type":"v1","addr":"172.21.15.134:6809","nonce":101352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":101352},{"type":"v1","addr":"172.21.15.134:6811","nonce":101352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":101352},{"type":"v1","addr":"172.21.15.134:6815","nonce":101352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":101352},{"type":"v1","addr":"172.21.15.134:6813","nonce":101352}]},"public_addr":"172.21.15.134:6809/101352","cluster_addr":"172.21.15.134:6811/101352","heartbeat_back_addr":"172.21.15.134:6815/101352","heartbeat_front_addr":"172.21.15.134:6813/101352","state":["exists","up"]},{"osd":1,"uuid":"af1b8ac9-0335-4550-aaf8-9f15545625e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":101351},{"type":"v1","addr":"172.21.15.134:6801","nonce":101351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":101351},{"type":"v1","addr":"172.21.15.134:6803","nonce":101351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":101351},{"type":"v1","addr":"172.21.15.134:6807","nonce":101351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":101351},{"type":"v1","addr":"172.21.15.134:6805","nonce":101351}]},"public_addr":"172.21.15.134:6801/101351","cluster_addr":"172.21.15.134:6803/101351","heartbeat_back_addr":"172.21.15.134:6807/101351","heartbeat_front_addr":"172.21.15.134:6805/101351","state":["exists","up"]},{"osd":2,"uuid":"5bbdd443-3ec5-40fb-86af-3e5a6ab1606a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":101353},{"type":"v1","addr":"172.21.15.134:6817","nonce":101353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":101353},{"type":"v1","addr":"172.21.15.134:6819","nonce":101353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":101353},{"type":"v1","addr":"172.21.15.134:6823","nonce":101353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":101353},{"type":"v1","addr":"172.21.15.134:6821","nonce":101353}]},"public_addr":"172.21.15.134:6817/101353","cluster_addr":"172.21.15.134:6819/101353","heartbeat_back_addr":"172.21.15.134:6823/101353","heartbeat_front_addr":"172.21.15.134:6821/101353","state":["exists","up"]},{"osd":3,"uuid":"795fd2fb-0e2e-4419-9b54-ea28ba0ee835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":101371},{"type":"v1","addr":"172.21.15.134:6825","nonce":101371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":101371},{"type":"v1","addr":"172.21.15.134:6827","nonce":101371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":101371},{"type":"v1","addr":"172.21.15.134:6831","nonce":101371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":101371},{"type":"v1","addr":"172.21.15.134:6829","nonce":101371}]},"public_addr":"172.21.15.134:6825/101371","cluster_addr":"172.21.15.134:6827/101371","heartbeat_back_addr":"172.21.15.134:6831/101371","heartbeat_front_addr":"172.21.15.134:6829/101371","state":["exists","up"]},{"osd":4,"uuid":"36005e5e-73a9-4d10-aba3-480165df1594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":101536},{"type":"v1","addr":"172.21.15.191:6817","nonce":101536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":101536},{"type":"v1","addr":"172.21.15.191:6819","nonce":101536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":101536},{"type":"v1","addr":"172.21.15.191:6823","nonce":101536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":101536},{"type":"v1","addr":"172.21.15.191:6821","nonce":101536}]},"public_addr":"172.21.15.191:6817/101536","cluster_addr":"172.21.15.191:6819/101536","heartbeat_back_addr":"172.21.15.191:6823/101536","heartbeat_front_addr":"172.21.15.191:6821/101536","state":["exists","up"]},{"osd":5,"uuid":"6f5b5e55-e69b-4097-a47c-cd7fc0681dbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":101534},{"type":"v1","addr":"172.21.15.191:6801","nonce":101534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":101534},{"type":"v1","addr":"172.21.15.191:6803","nonce":101534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":101534},{"type":"v1","addr":"172.21.15.191:6807","nonce":101534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":101534},{"type":"v1","addr":"172.21.15.191:6805","nonce":101534}]},"public_addr":"172.21.15.191:6801/101534","cluster_addr":"172.21.15.191:6803/101534","heartbeat_back_addr":"172.21.15.191:6807/101534","heartbeat_front_addr":"172.21.15.191:6805/101534","state":["exists","up"]},{"osd":6,"uuid":"074077c9-fc46-4f79-8368-5a96eba6da06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":101545},{"type":"v1","addr":"172.21.15.191:6825","nonce":101545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":101545},{"type":"v1","addr":"172.21.15.191:6827","nonce":101545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":101545},{"type":"v1","addr":"172.21.15.191:6831","nonce":101545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":101545},{"type":"v1","addr":"172.21.15.191:6829","nonce":101545}]},"public_addr":"172.21.15.191:6825/101545","cluster_addr":"172.21.15.191:6827/101545","heartbeat_back_addr":"172.21.15.191:6831/101545","heartbeat_front_addr":"172.21.15.191:6829/101545","state":["exists","up"]},{"osd":7,"uuid":"112889bf-b637-4500-a30d-03e23e8a3267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":101535},{"type":"v1","addr":"172.21.15.191:6809","nonce":101535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":101535},{"type":"v1","addr":"172.21.15.191:6811","nonce":101535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":101535},{"type":"v1","addr":"172.21.15.191:6815","nonce":101535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":101535},{"type":"v1","addr":"172.21.15.191:6813","nonce":101535}]},"public_addr":"172.21.15.191:6809/101535","cluster_addr":"172.21.15.191:6811/101535","heartbeat_back_addr":"172.21.15.191:6815/101535","heartbeat_front_addr":"172.21.15.191:6813/101535","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.141813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:03.885025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.127132+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.180968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.193639+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.201380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.195434+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.198016+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:17:17.173 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.169+0000 7f02f8eb6700 1 -- 172.21.15.134:0/3644955231 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f02d0066580 msgr2=0x7f02d0068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.173 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.169+0000 7f02f8eb6700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f02d0066580 0x7f02d0068a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f02e40090b0 tx=0x7f02e401f040 comp rx=0 tx=0).stop 2022-11-27T16:17:17.174 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.169+0000 7f02f8eb6700 1 -- 172.21.15.134:0/3644955231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f41563c0 msgr2=0x7f02f40b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.174 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.169+0000 7f02f8eb6700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f41563c0 0x7f02f40b3850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02f40cddb0 tx=0x7f02ec00a5b0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.175 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.170+0000 7f02f8eb6700 1 -- 172.21.15.134:0/3644955231 shutdown_connections 2022-11-27T16:17:17.175 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.170+0000 7f02f8eb6700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f41563c0 0x7f02f40b3850 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.175 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.170+0000 7f02f8eb6700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f02d0066580 0x7f02d0068a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.176 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.170+0000 7f02f8eb6700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02d8003bc0 0x7f02f40b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.176 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.170+0000 7f02f8eb6700 1 --2- 172.21.15.134:0/3644955231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02f40aff40 0x7f02f4153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.177 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.170+0000 7f02f8eb6700 1 -- 172.21.15.134:0/3644955231 >> 172.21.15.134:0/3644955231 conn(0x7f02f406a500 msgr2=0x7f02f4070bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:17.177 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.170+0000 7f02f8eb6700 1 -- 172.21.15.134:0/3644955231 shutdown_connections 2022-11-27T16:17:17.177 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.170+0000 7f02f8eb6700 1 -- 172.21.15.134:0/3644955231 wait complete. 2022-11-27T16:17:17.184 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-11-27T16:17:09.769201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T16:17:14.856029+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T16:17:17.185 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T16:17:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.306+0000 7f0f54d7c700 1 Processor -- start 2022-11-27T16:17:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.306+0000 7f0f54d7c700 1 -- start start 2022-11-27T16:17:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f54d7c700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0f5015bb10 0x7f0f500693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f54d7c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f5015bee0 0x7f0f5006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f54d7c700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5006c050 con 0x7f0f5006cd70 2022-11-27T16:17:17.319 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f54d7c700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f50069c40 con 0x7f0f5015bb10 2022-11-27T16:17:17.319 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f54d7c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50069f20 con 0x7f0f5015bee0 2022-11-27T16:17:17.320 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f4e59c700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0f5015bb10 0x7f0f500693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.320 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f4ed9d700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f0f5006cd70 0x7f0f5006d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56112/0 (socket says 172.21.15.134:56112) 2022-11-27T16:17:17.320 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f4ed9d700 1 -- 172.21.15.134:0/3259332655 learned_addr learned my addr 172.21.15.134:0/3259332655 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:17:17.321 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f4dd9b700 1 --2- 172.21.15.134:0/3259332655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f5015bee0 0x7f0f5006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.321 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f4e59c700 1 -- 172.21.15.134:0/3259332655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f5015bee0 msgr2=0x7f0f5006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f4e59c700 1 --2- 172.21.15.134:0/3259332655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f5015bee0 0x7f0f5006c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f4e59c700 1 -- 172.21.15.134:0/3259332655 >> v1:172.21.15.134:6789/0 conn(0x7f0f5006cd70 legacy=0x7f0f5006d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f4e59c700 1 -- 172.21.15.134:0/3259332655 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5006a200 con 0x7f0f5015bb10 2022-11-27T16:17:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f4d59a700 1 -- 172.21.15.134:0/3259332655 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915249570 0 0) 0x7f0f5006c050 con 0x7f0f5006cd70 2022-11-27T16:17:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.307+0000 7f0f4dd9b700 1 --2- 172.21.15.134:0/3259332655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f5015bee0 0x7f0f5006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:17:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f4e59c700 1 --2- 172.21.15.134:0/3259332655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5015bb10 0x7f0f500693f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009840 tx=0x7f0f44009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d874bc7e4e36a696 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f4d59a700 1 -- 172.21.15.134:0/3259332655 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44011830 con 0x7f0f5015bb10 2022-11-27T16:17:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f4d59a700 1 -- 172.21.15.134:0/3259332655 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f44011990 con 0x7f0f5015bb10 2022-11-27T16:17:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f4d59a700 1 -- 172.21.15.134:0/3259332655 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44011d20 con 0x7f0f5015bb10 2022-11-27T16:17:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f54d7c700 1 -- 172.21.15.134:0/3259332655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5015bb10 msgr2=0x7f0f500693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f54d7c700 1 --2- 172.21.15.134:0/3259332655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5015bb10 0x7f0f500693f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009840 tx=0x7f0f44009b50 comp rx=0 tx=0).stop 2022-11-27T16:17:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f54d7c700 1 -- 172.21.15.134:0/3259332655 shutdown_connections 2022-11-27T16:17:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f54d7c700 1 --2- 172.21.15.134:0/3259332655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5015bb10 0x7f0f500693f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f54d7c700 1 --2- 172.21.15.134:0/3259332655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f5015bee0 0x7f0f5006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.327 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f54d7c700 1 -- 172.21.15.134:0/3259332655 >> 172.21.15.134:0/3259332655 conn(0x7f0f500b16c0 msgr2=0x7f0f500cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:17.327 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f54d7c700 1 -- 172.21.15.134:0/3259332655 shutdown_connections 2022-11-27T16:17:17.328 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.308+0000 7f0f54d7c700 1 -- 172.21.15.134:0/3259332655 wait complete. 2022-11-27T16:17:17.328 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f54d7c700 1 Processor -- start 2022-11-27T16:17:17.328 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f54d7c700 1 -- start start 2022-11-27T16:17:17.329 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f54d7c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5006cd70 0x7f0f500c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.329 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f54d7c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f5015bb10 0x7f0f50145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.330 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f54d7c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f5015bee0 0x7f0f500c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.330 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f54d7c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f501470f0 con 0x7f0f5006cd70 2022-11-27T16:17:17.330 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f54d7c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500c6890 con 0x7f0f5015bb10 2022-11-27T16:17:17.331 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f54d7c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50146240 con 0x7f0f5015bee0 2022-11-27T16:17:17.331 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f4e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5006cd70 0x7f0f500c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.331 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f4e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5006cd70 0x7f0f500c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38042/0 (socket says 172.21.15.134:38042) 2022-11-27T16:17:17.332 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.310+0000 7f0f4e59c700 1 -- 172.21.15.134:0/1310844499 learned_addr learned my addr 172.21.15.134:0/1310844499 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:17.332 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f4e59c700 1 -- 172.21.15.134:0/1310844499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f5015bee0 msgr2=0x7f0f500c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.333 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f4e59c700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f5015bee0 0x7f0f500c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.333 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f4dd9b700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f5015bb10 0x7f0f50145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.333 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f4e59c700 1 -- 172.21.15.134:0/1310844499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f5015bb10 msgr2=0x7f0f50145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.334 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f4e59c700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f5015bb10 0x7f0f50145d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.334 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f4e59c700 1 -- 172.21.15.134:0/1310844499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f440094a0 con 0x7f0f5006cd70 2022-11-27T16:17:17.335 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f4e59c700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5006cd70 0x7f0f500c6350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009b90 tx=0x7f0f44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.335 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f3effd700 1 -- 172.21.15.134:0/1310844499 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f440036a0 con 0x7f0f5006cd70 2022-11-27T16:17:17.335 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f3effd700 1 -- 172.21.15.134:0/1310844499 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4401f070 con 0x7f0f5006cd70 2022-11-27T16:17:17.336 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f54d7c700 1 -- 172.21.15.134:0/1310844499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f500c81c0 con 0x7f0f5006cd70 2022-11-27T16:17:17.336 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f3effd700 1 -- 172.21.15.134:0/1310844499 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44003a40 con 0x7f0f5006cd70 2022-11-27T16:17:17.336 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.311+0000 7f0f54d7c700 1 -- 172.21.15.134:0/1310844499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f50146520 con 0x7f0f5006cd70 2022-11-27T16:17:17.337 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.312+0000 7f0f54d7c700 1 -- 172.21.15.134:0/1310844499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f50065ac0 con 0x7f0f5006cd70 2022-11-27T16:17:17.337 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.312+0000 7f0f3effd700 1 -- 172.21.15.134:0/1310844499 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f0f4400f050 con 0x7f0f5006cd70 2022-11-27T16:17:17.338 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.313+0000 7f0f3effd700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f0f2c066590 0x7f0f2c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.338 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.313+0000 7f0f3effd700 1 -- 172.21.15.134:0/1310844499 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0f44021070 con 0x7f0f5006cd70 2022-11-27T16:17:17.338 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.313+0000 7f0f4dd9b700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f0f2c066590 0x7f0f2c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.339 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.313+0000 7f0f4dd9b700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f0f2c066590 0x7f0f2c068a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0f500cafa0 tx=0x7f0f40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.340 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.317+0000 7f0f3effd700 1 -- 172.21.15.134:0/1310844499 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f0f440528d0 con 0x7f0f5006cd70 2022-11-27T16:17:17.501 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.499+0000 7f0f54d7c700 1 -- 172.21.15.134:0/1310844499 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f5006c310 con 0x7f0f5006cd70 2022-11-27T16:17:17.501 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.499+0000 7f0f3effd700 1 -- 172.21.15.134:0/1310844499 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f4402b0a0 con 0x7f0f5006cd70 2022-11-27T16:17:17.502 INFO:teuthology.orchestra.run.smithi134.stdout:pg_num: 1 2022-11-27T16:17:17.504 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.501+0000 7f0f54d7c700 1 -- 172.21.15.134:0/1310844499 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f0f2c066590 msgr2=0x7f0f2c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.504 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.501+0000 7f0f54d7c700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f0f2c066590 0x7f0f2c068a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0f500cafa0 tx=0x7f0f40008040 comp rx=0 tx=0).stop 2022-11-27T16:17:17.504 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.501+0000 7f0f54d7c700 1 -- 172.21.15.134:0/1310844499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5006cd70 msgr2=0x7f0f500c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.504 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.501+0000 7f0f54d7c700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5006cd70 0x7f0f500c6350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009b90 tx=0x7f0f44004750 comp rx=0 tx=0).stop 2022-11-27T16:17:17.505 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.501+0000 7f0f54d7c700 1 -- 172.21.15.134:0/1310844499 shutdown_connections 2022-11-27T16:17:17.505 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.501+0000 7f0f54d7c700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f5015bee0 0x7f0f500c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.505 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.501+0000 7f0f54d7c700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f5006cd70 0x7f0f500c6350 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.505 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.501+0000 7f0f54d7c700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f0f2c066590 0x7f0f2c068a40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.505 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.501+0000 7f0f54d7c700 1 --2- 172.21.15.134:0/1310844499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f5015bb10 0x7f0f50145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.506 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.501+0000 7f0f54d7c700 1 -- 172.21.15.134:0/1310844499 >> 172.21.15.134:0/1310844499 conn(0x7f0f500b16c0 msgr2=0x7f0f500c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:17.506 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.502+0000 7f0f54d7c700 1 -- 172.21.15.134:0/1310844499 shutdown_connections 2022-11-27T16:17:17.506 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.502+0000 7f0f54d7c700 1 -- 172.21.15.134:0/1310844499 wait complete. 2022-11-27T16:17:17.515 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T16:17:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.637+0000 7fbeeb099700 1 Processor -- start 2022-11-27T16:17:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.637+0000 7fbeeb099700 1 -- start start 2022-11-27T16:17:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbeeb099700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbee40b2dc0 0x7fbee40b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbeeb099700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee406d120 0x7fbee406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbeeb099700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee40b1ca0 con 0x7fbee41598c0 2022-11-27T16:17:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbeeb099700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee40cc830 con 0x7fbee40b2dc0 2022-11-27T16:17:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbeeb099700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee40b3770 con 0x7fbee406d120 2022-11-27T16:17:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbee9636700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee406d120 0x7fbee406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbee3fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbee40b2dc0 0x7fbee40b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbee8e35700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fbee41598c0 0x7fbee40d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.134:48666/0 (socket says 172.21.15.134:48666) 2022-11-27T16:17:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbee3fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbee40b2dc0 0x7fbee40b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.134:54446/0 (socket says 172.21.15.134:54446) 2022-11-27T16:17:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbee9636700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee406d120 0x7fbee406d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.134:41836/0 (socket says 172.21.15.134:41836) 2022-11-27T16:17:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbee8e35700 1 -- 172.21.15.134:0/2212869332 learned_addr learned my addr 172.21.15.134:0/2212869332 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:17:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbee3fff700 1 -- 172.21.15.134:0/2212869332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee406d120 msgr2=0x7fbee406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbee3fff700 1 --2- 172.21.15.134:0/2212869332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee406d120 0x7fbee406d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.638+0000 7fbee3fff700 1 -- 172.21.15.134:0/2212869332 >> v1:172.21.15.191:6789/0 conn(0x7fbee41598c0 legacy=0x7fbee40d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbee3fff700 1 -- 172.21.15.134:0/2212869332 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee40d0cb0 con 0x7fbee40b2dc0 2022-11-27T16:17:17.646 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbee9636700 1 --2- 172.21.15.134:0/2212869332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee406d120 0x7fbee406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:17:17.646 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbee3fff700 1 --2- 172.21.15.134:0/2212869332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee40b2dc0 0x7fbee40b3230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009840 tx=0x7fbed0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e868f4a4e3a44a7 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.646 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbee37fe700 1 -- 172.21.15.134:0/2212869332 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0011830 con 0x7fbee40b2dc0 2022-11-27T16:17:17.646 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbee37fe700 1 -- 172.21.15.134:0/2212869332 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed0011990 con 0x7fbee40b2dc0 2022-11-27T16:17:17.646 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbee37fe700 1 -- 172.21.15.134:0/2212869332 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0011d20 con 0x7fbee40b2dc0 2022-11-27T16:17:17.647 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbeeb099700 1 -- 172.21.15.134:0/2212869332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee40b2dc0 msgr2=0x7fbee40b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.647 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbeeb099700 1 --2- 172.21.15.134:0/2212869332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee40b2dc0 0x7fbee40b3230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009840 tx=0x7fbed0009b50 comp rx=0 tx=0).stop 2022-11-27T16:17:17.647 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbeeb099700 1 -- 172.21.15.134:0/2212869332 shutdown_connections 2022-11-27T16:17:17.647 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbeeb099700 1 --2- 172.21.15.134:0/2212869332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee406d120 0x7fbee406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.648 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbeeb099700 1 --2- 172.21.15.134:0/2212869332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee40b2dc0 0x7fbee40b3230 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.648 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbeeb099700 1 -- 172.21.15.134:0/2212869332 >> 172.21.15.134:0/2212869332 conn(0x7fbee4071060 msgr2=0x7fbee4071430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:17.648 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbeeb099700 1 -- 172.21.15.134:0/2212869332 shutdown_connections 2022-11-27T16:17:17.648 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.639+0000 7fbeeb099700 1 -- 172.21.15.134:0/2212869332 wait complete. 2022-11-27T16:17:17.649 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.641+0000 7fbeeb099700 1 Processor -- start 2022-11-27T16:17:17.649 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.641+0000 7fbeeb099700 1 -- start start 2022-11-27T16:17:17.652 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.641+0000 7fbeeb099700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee406d120 0x7fbee41f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.652 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.641+0000 7fbeeb099700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee40b2dc0 0x7fbee40c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.641+0000 7fbeeb099700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee41598c0 0x7fbee406b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.641+0000 7fbeeb099700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee4157fe0 con 0x7fbee41598c0 2022-11-27T16:17:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbeeb099700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee41576d0 con 0x7fbee406d120 2022-11-27T16:17:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbeeb099700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee40c9d90 con 0x7fbee40b2dc0 2022-11-27T16:17:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee9636700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee41598c0 0x7fbee406b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee9636700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee41598c0 0x7fbee406b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38058/0 (socket says 172.21.15.134:38058) 2022-11-27T16:17:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee9636700 1 -- 172.21.15.134:0/675495249 learned_addr learned my addr 172.21.15.134:0/675495249 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee3fff700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee40b2dc0 0x7fbee40c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee8e35700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee406d120 0x7fbee41f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee9636700 1 -- 172.21.15.134:0/675495249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee40b2dc0 msgr2=0x7fbee40c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee9636700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee40b2dc0 0x7fbee40c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee9636700 1 -- 172.21.15.134:0/675495249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee406d120 msgr2=0x7fbee41f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee9636700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee406d120 0x7fbee41f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee9636700 1 -- 172.21.15.134:0/675495249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc009a70 con 0x7fbee41598c0 2022-11-27T16:17:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee3fff700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee40b2dc0 0x7fbee40c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:17:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee8e35700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee406d120 0x7fbee41f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:17:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee9636700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee41598c0 0x7fbee406b3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009f00 tx=0x7fbedc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee17fa700 1 -- 172.21.15.134:0/675495249 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc009d70 con 0x7fbee41598c0 2022-11-27T16:17:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee17fa700 1 -- 172.21.15.134:0/675495249 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc00f040 con 0x7fbee41598c0 2022-11-27T16:17:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbeeb099700 1 -- 172.21.15.134:0/675495249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed00094a0 con 0x7fbee41598c0 2022-11-27T16:17:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbee17fa700 1 -- 172.21.15.134:0/675495249 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00cd70 con 0x7fbee41598c0 2022-11-27T16:17:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.642+0000 7fbeeb099700 1 -- 172.21.15.134:0/675495249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee4157920 con 0x7fbee41598c0 2022-11-27T16:17:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.643+0000 7fbeeb099700 1 -- 172.21.15.134:0/675495249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee40c9ed0 con 0x7fbee41598c0 2022-11-27T16:17:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.644+0000 7fbee17fa700 1 -- 172.21.15.134:0/675495249 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fbedc014480 con 0x7fbee41598c0 2022-11-27T16:17:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.644+0000 7fbee17fa700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fbecc066590 0x7fbecc068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.644+0000 7fbee17fa700 1 -- 172.21.15.134:0/675495249 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbedc083c20 con 0x7fbee41598c0 2022-11-27T16:17:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.644+0000 7fbee8e35700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fbecc066590 0x7fbecc068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.644+0000 7fbee8e35700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fbecc066590 0x7fbecc068a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005910 tx=0x7fbed8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.648+0000 7fbee17fa700 1 -- 172.21.15.134:0/675495249 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fbee40c9ed0 con 0x7fbee41598c0 2022-11-27T16:17:17.831 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.829+0000 7fbeeb099700 1 -- 172.21.15.134:0/675495249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbee40c9ed0 con 0x7fbee41598c0 2022-11-27T16:17:17.831 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.829+0000 7fbee17fa700 1 -- 172.21.15.134:0/675495249 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbee40c9ed0 con 0x7fbee41598c0 2022-11-27T16:17:17.832 INFO:teuthology.orchestra.run.smithi134.stdout:pg_num: 8 2022-11-27T16:17:17.833 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.831+0000 7fbeeb099700 1 -- 172.21.15.134:0/675495249 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fbecc066590 msgr2=0x7fbecc068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.834 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.831+0000 7fbeeb099700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fbecc066590 0x7fbecc068a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005910 tx=0x7fbed8009040 comp rx=0 tx=0).stop 2022-11-27T16:17:17.834 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.831+0000 7fbeeb099700 1 -- 172.21.15.134:0/675495249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee41598c0 msgr2=0x7fbee406b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.834 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.831+0000 7fbeeb099700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee41598c0 0x7fbee406b3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009f00 tx=0x7fbedc00c3a0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.834 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.832+0000 7fbeeb099700 1 -- 172.21.15.134:0/675495249 shutdown_connections 2022-11-27T16:17:17.835 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.832+0000 7fbeeb099700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee41598c0 0x7fbee406b3f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.835 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.832+0000 7fbeeb099700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fbecc066590 0x7fbecc068a40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.835 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.832+0000 7fbeeb099700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbee406d120 0x7fbee41f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.835 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.832+0000 7fbeeb099700 1 --2- 172.21.15.134:0/675495249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbee40b2dc0 0x7fbee40c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.836 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.832+0000 7fbeeb099700 1 -- 172.21.15.134:0/675495249 >> 172.21.15.134:0/675495249 conn(0x7fbee4071060 msgr2=0x7fbee40cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:17.836 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.832+0000 7fbeeb099700 1 -- 172.21.15.134:0/675495249 shutdown_connections 2022-11-27T16:17:17.836 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.832+0000 7fbeeb099700 1 -- 172.21.15.134:0/675495249 wait complete. 2022-11-27T16:17:17.845 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T16:17:17.967 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.965+0000 7f652b04c700 1 Processor -- start 2022-11-27T16:17:17.968 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.965+0000 7f652b04c700 1 -- start start 2022-11-27T16:17:17.975 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f652b04c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f65240cf2c0 0x7f652406a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.976 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f652b04c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65240cfae0 0x7f65240cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.976 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f652b04c700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652406b8f0 con 0x7f65240cf690 2022-11-27T16:17:17.976 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f652b04c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652406c490 con 0x7f65240cf2c0 2022-11-27T16:17:17.977 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f652b04c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652406aa40 con 0x7f65240cfae0 2022-11-27T16:17:17.977 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f6523fff700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f65240cf690 0x7f652406bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56138/0 (socket says 172.21.15.134:56138) 2022-11-27T16:17:17.977 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f6523fff700 1 -- 172.21.15.134:0/294530528 learned_addr learned my addr 172.21.15.134:0/294530528 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:17:17.977 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352241852 0 0) 0x7f652406b8f0 con 0x7f65240cf690 2022-11-27T16:17:17.978 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65040035b0 con 0x7f65240cf690 2022-11-27T16:17:17.978 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f6528de8700 1 --2- 172.21.15.134:0/294530528 >> v2:172.21.15.191:3300/0 conn(0x7f65240cf2c0 0x7f652406a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.978 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65295e9700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65240cfae0 0x7f65240cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.978 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1812848712 0 0) 0x7f65040035b0 con 0x7f65240cf690 2022-11-27T16:17:17.979 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65240cfae0 msgr2=0x7f65240cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:17.979 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65240cfae0 0x7f65240cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.979 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 >> v2:172.21.15.191:3300/0 conn(0x7f65240cf2c0 msgr2=0x7f652406a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:17.979 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 --2- 172.21.15.134:0/294530528 >> v2:172.21.15.191:3300/0 conn(0x7f65240cf2c0 0x7f652406a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.980 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652406ace0 con 0x7f65240cf690 2022-11-27T16:17:17.980 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2726658286 0 0) 0x7f650c001670 con 0x7f65240cf690 2022-11-27T16:17:17.980 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 >> v1:172.21.15.134:6789/0 conn(0x7f65240cf690 legacy=0x7f652406bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.980 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6504003a00 0x7f6504003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.981 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6504004430 0x7f65040048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.981 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6504004e80 0x7f65040052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.981 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652406c490 con 0x7f6504004e80 2022-11-27T16:17:17.981 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652406aa40 con 0x7f6504003a00 2022-11-27T16:17:17.981 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650400bd40 con 0x7f6504004430 2022-11-27T16:17:17.982 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f6528de8700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6504003a00 0x7f6504003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.982 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f65295e9700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6504004e80 0x7f65040052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.982 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.967+0000 7f6523fff700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6504004430 0x7f65040048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.982 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f65295e9700 1 -- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6504004430 msgr2=0x7f65040048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:17.983 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f65295e9700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6504004430 0x7f65040048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.983 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f65295e9700 1 -- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6504003a00 msgr2=0x7f6504003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:17.983 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f65295e9700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6504003a00 0x7f6504003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.983 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f65295e9700 1 -- 172.21.15.134:0/294530528 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040035b0 con 0x7f6504004e80 2022-11-27T16:17:17.983 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f65295e9700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6504004e80 0x7f65040052f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f651c009f90 tx=0x7f6504005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974038f195b670ff server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.984 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c00ec30 con 0x7f6504004e80 2022-11-27T16:17:17.984 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c00f040 con 0x7f6504004e80 2022-11-27T16:17:17.984 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f65237fe700 1 -- 172.21.15.134:0/294530528 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c014480 con 0x7f6504004e80 2022-11-27T16:17:17.984 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f652b04c700 1 -- 172.21.15.134:0/294530528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6504004e80 msgr2=0x7f65040052f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.985 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f652b04c700 1 --2- 172.21.15.134:0/294530528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6504004e80 0x7f65040052f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f651c009f90 tx=0x7f6504005880 comp rx=0 tx=0).stop 2022-11-27T16:17:17.985 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.968+0000 7f65295e9700 1 -- 172.21.15.134:0/294530528 reap_dead start 2022-11-27T16:17:17.985 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.969+0000 7f652b04c700 1 -- 172.21.15.134:0/294530528 shutdown_connections 2022-11-27T16:17:17.985 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.969+0000 7f652b04c700 1 -- 172.21.15.134:0/294530528 >> 172.21.15.134:0/294530528 conn(0x7f652406de60 msgr2=0x7f65240693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:17.986 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.969+0000 7f652b04c700 1 -- 172.21.15.134:0/294530528 shutdown_connections 2022-11-27T16:17:17.986 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.969+0000 7f652b04c700 1 -- 172.21.15.134:0/294530528 wait complete. 2022-11-27T16:17:17.986 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.970+0000 7f652b04c700 1 Processor -- start 2022-11-27T16:17:17.986 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.970+0000 7f652b04c700 1 -- start start 2022-11-27T16:17:17.986 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f652b04c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65241f1b70 0x7f65240b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.987 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f652b04c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f652415d940 0x7f652415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.987 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f652b04c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65240af890 0x7f65240afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.987 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f652b04c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65240b47f0 con 0x7f65241f1b70 2022-11-27T16:17:17.987 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f652b04c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652415e2f0 con 0x7f65240af890 2022-11-27T16:17:17.988 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f652b04c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65240b3e20 con 0x7f652415d940 2022-11-27T16:17:17.988 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f6528de8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65241f1b70 0x7f65240b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.988 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f6528de8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65241f1b70 0x7f65240b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38076/0 (socket says 172.21.15.134:38076) 2022-11-27T16:17:17.988 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f6528de8700 1 -- 172.21.15.134:0/2743123105 learned_addr learned my addr 172.21.15.134:0/2743123105 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:17.988 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f65295e9700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65240af890 0x7f65240afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.989 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f6523fff700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f652415d940 0x7f652415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.989 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f6528de8700 1 -- 172.21.15.134:0/2743123105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f652415d940 msgr2=0x7f652415ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.989 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f6528de8700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f652415d940 0x7f652415ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.989 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f6528de8700 1 -- 172.21.15.134:0/2743123105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65240af890 msgr2=0x7f65240afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:17.990 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f6528de8700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65240af890 0x7f65240afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:17.990 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f6528de8700 1 -- 172.21.15.134:0/2743123105 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65240b08a0 con 0x7f65241f1b70 2022-11-27T16:17:17.990 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f6528de8700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65241f1b70 0x7f65240b42b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6514011b00 tx=0x7f6514011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.990 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f65217fa700 1 -- 172.21.15.134:0/2743123105 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400f8e0 con 0x7f65241f1b70 2022-11-27T16:17:17.990 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f65217fa700 1 -- 172.21.15.134:0/2743123105 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651400fa40 con 0x7f65241f1b70 2022-11-27T16:17:17.991 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f652b04c700 1 -- 172.21.15.134:0/2743123105 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65240b05c0 con 0x7f65241f1b70 2022-11-27T16:17:17.991 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f65217fa700 1 -- 172.21.15.134:0/2743123105 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400fdf0 con 0x7f65241f1b70 2022-11-27T16:17:17.991 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.971+0000 7f652b04c700 1 -- 172.21.15.134:0/2743123105 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65240b3ab0 con 0x7f65241f1b70 2022-11-27T16:17:17.991 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.972+0000 7f652b04c700 1 -- 172.21.15.134:0/2743123105 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65240b4100 con 0x7f65241f1b70 2022-11-27T16:17:17.992 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.973+0000 7f65217fa700 1 -- 172.21.15.134:0/2743123105 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f651401d400 con 0x7f65241f1b70 2022-11-27T16:17:17.992 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.973+0000 7f65217fa700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f6508066590 0x7f6508068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:17.992 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.974+0000 7f6523fff700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f6508066590 0x7f6508068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:17.992 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.974+0000 7f65217fa700 1 -- 172.21.15.134:0/2743123105 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6514084210 con 0x7f65241f1b70 2022-11-27T16:17:17.992 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.974+0000 7f6523fff700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f6508066590 0x7f6508068a40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f650c001cf0 tx=0x7f650c007980 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:17.993 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:17.977+0000 7f65217fa700 1 -- 172.21.15.134:0/2743123105 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f6514051cb0 con 0x7f65241f1b70 2022-11-27T16:17:18.200 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.198+0000 7f652b04c700 1 -- 172.21.15.134:0/2743123105 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6524073030 con 0x7f65241f1b70 2022-11-27T16:17:18.201 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.199+0000 7f65217fa700 1 -- 172.21.15.134:0/2743123105 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7f65140552d0 con 0x7f65241f1b70 2022-11-27T16:17:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:17:18.202 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1121142543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1121142543},{"type":"v1","addr":"172.21.15.191:6834","nonce":1121142543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2445797197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2445797197},{"type":"v1","addr":"172.21.15.134:6836","nonce":2445797197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1545939201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1545939201},{"type":"v1","addr":"172.21.15.134:6837","nonce":1545939201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3414433274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3414433274},{"type":"v1","addr":"172.21.15.191:6835","nonce":3414433274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2022-11-27T16:17:18.203 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 -- 172.21.15.134:0/2743123105 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f6508066590 msgr2=0x7f6508068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.204 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f6508066590 0x7f6508068a40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f650c001cf0 tx=0x7f650c007980 comp rx=0 tx=0).stop 2022-11-27T16:17:18.204 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 -- 172.21.15.134:0/2743123105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65241f1b70 msgr2=0x7f65240b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.204 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65241f1b70 0x7f65240b42b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6514011b00 tx=0x7f6514011e10 comp rx=0 tx=0).stop 2022-11-27T16:17:18.205 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 -- 172.21.15.134:0/2743123105 shutdown_connections 2022-11-27T16:17:18.205 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f6508066590 0x7f6508068a40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.205 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65240af890 0x7f65240afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.205 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65241f1b70 0x7f65240b42b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.206 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 --2- 172.21.15.134:0/2743123105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f652415d940 0x7f652415ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.206 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 -- 172.21.15.134:0/2743123105 >> 172.21.15.134:0/2743123105 conn(0x7f652406de60 msgr2=0x7f652415b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:18.206 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 -- 172.21.15.134:0/2743123105 shutdown_connections 2022-11-27T16:17:18.206 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.201+0000 7f652b04c700 1 -- 172.21.15.134:0/2743123105 wait complete. 2022-11-27T16:17:18.207 INFO:teuthology.orchestra.run.smithi134.stderr:dumped fsmap epoch 3 2022-11-27T16:17:18.215 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:17:18.337 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.335+0000 7f060c219700 1 Processor -- start 2022-11-27T16:17:18.337 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.335+0000 7f060c219700 1 -- start start 2022-11-27T16:17:18.340 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f060c219700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0604158c40 0x7f0604070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.341 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f060c219700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f06040cd020 0x7f06040cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.341 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f060c219700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0604070070 con 0x7f060415d3a0 2022-11-27T16:17:18.341 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f060c219700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06040cd9a0 con 0x7f0604158c40 2022-11-27T16:17:18.341 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f060c219700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060406c780 con 0x7f06040cd020 2022-11-27T16:17:18.342 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f0609fb5700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0604158c40 0x7f0604070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.342 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f06097b4700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f060415d3a0 0x7f060406c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56158/0 (socket says 172.21.15.134:56158) 2022-11-27T16:17:18.342 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f06097b4700 1 -- 172.21.15.134:0/1155028100 learned_addr learned my addr 172.21.15.134:0/1155028100 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:17:18.342 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f060a7b6700 1 --2- 172.21.15.134:0/1155028100 >> v2:172.21.15.191:3300/0 conn(0x7f06040cd020 0x7f06040cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.342 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f0609fb5700 1 -- 172.21.15.134:0/1155028100 >> v2:172.21.15.191:3300/0 conn(0x7f06040cd020 msgr2=0x7f06040cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:18.343 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f0609fb5700 1 --2- 172.21.15.134:0/1155028100 >> v2:172.21.15.191:3300/0 conn(0x7f06040cd020 0x7f06040cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.343 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f0609fb5700 1 -- 172.21.15.134:0/1155028100 >> v1:172.21.15.134:6789/0 conn(0x7f060415d3a0 legacy=0x7f060406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.343 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f0609fb5700 1 -- 172.21.15.134:0/1155028100 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604159010 con 0x7f0604158c40 2022-11-27T16:17:18.343 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f0608fb3700 1 -- 172.21.15.134:0/1155028100 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365548000 0 0) 0x7f0604070070 con 0x7f060415d3a0 2022-11-27T16:17:18.344 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f060a7b6700 1 --2- 172.21.15.134:0/1155028100 >> v2:172.21.15.191:3300/0 conn(0x7f06040cd020 0x7f06040cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:17:18.344 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f0609fb5700 1 --2- 172.21.15.134:0/1155028100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0604158c40 0x7f0604070da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0604072990 tx=0x7f0600009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164de5744eef49b5 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:18.344 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.336+0000 7f0608fb3700 1 -- 172.21.15.134:0/1155028100 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600011830 con 0x7f0604158c40 2022-11-27T16:17:18.344 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.337+0000 7f0608fb3700 1 -- 172.21.15.134:0/1155028100 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0600011990 con 0x7f0604158c40 2022-11-27T16:17:18.345 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.337+0000 7f0608fb3700 1 -- 172.21.15.134:0/1155028100 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600011d20 con 0x7f0604158c40 2022-11-27T16:17:18.345 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.337+0000 7f060c219700 1 -- 172.21.15.134:0/1155028100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0604158c40 msgr2=0x7f0604070da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.345 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.337+0000 7f060c219700 1 --2- 172.21.15.134:0/1155028100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0604158c40 0x7f0604070da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0604072990 tx=0x7f0600009b20 comp rx=0 tx=0).stop 2022-11-27T16:17:18.345 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.337+0000 7f060c219700 1 -- 172.21.15.134:0/1155028100 shutdown_connections 2022-11-27T16:17:18.345 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.337+0000 7f060c219700 1 --2- 172.21.15.134:0/1155028100 >> v2:172.21.15.191:3300/0 conn(0x7f06040cd020 0x7f06040cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.346 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.337+0000 7f060c219700 1 --2- 172.21.15.134:0/1155028100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0604158c40 0x7f0604070da0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.346 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.337+0000 7f060c219700 1 -- 172.21.15.134:0/1155028100 >> 172.21.15.134:0/1155028100 conn(0x7f060406a500 msgr2=0x7f060406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:18.346 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.337+0000 7f060c219700 1 -- 172.21.15.134:0/1155028100 shutdown_connections 2022-11-27T16:17:18.346 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.337+0000 7f060c219700 1 -- 172.21.15.134:0/1155028100 wait complete. 2022-11-27T16:17:18.347 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.339+0000 7f060c219700 1 Processor -- start 2022-11-27T16:17:18.347 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.339+0000 7f060c219700 1 -- start start 2022-11-27T16:17:18.350 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f060c219700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06040cd020 0x7f0604157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.351 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f060c219700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0604158c40 0x7f06041591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.351 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f060c219700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060415d3a0 0x7f0604152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.351 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f060c219700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0604155450 con 0x7f06040cd020 2022-11-27T16:17:18.351 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f060c219700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0604159d20 con 0x7f0604158c40 2022-11-27T16:17:18.352 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f060c219700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06041556f0 con 0x7f060415d3a0 2022-11-27T16:17:18.352 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f0609fb5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06040cd020 0x7f0604157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.352 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f0609fb5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06040cd020 0x7f0604157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38090/0 (socket says 172.21.15.134:38090) 2022-11-27T16:17:18.352 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f0609fb5700 1 -- 172.21.15.134:0/2665147618 learned_addr learned my addr 172.21.15.134:0/2665147618 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:18.352 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f06097b4700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0604158c40 0x7f06041591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f060a7b6700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060415d3a0 0x7f0604152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f0609fb5700 1 -- 172.21.15.134:0/2665147618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060415d3a0 msgr2=0x7f0604152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f0609fb5700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060415d3a0 0x7f0604152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f0609fb5700 1 -- 172.21.15.134:0/2665147618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0604158c40 msgr2=0x7f06041591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f0609fb5700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0604158c40 0x7f06041591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f0609fb5700 1 -- 172.21.15.134:0/2665147618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000094a0 con 0x7f06040cd020 2022-11-27T16:17:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f06097b4700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0604158c40 0x7f06041591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:17:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f060a7b6700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060415d3a0 0x7f0604152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:17:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f0609fb5700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06040cd020 0x7f0604157210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0600000c00 tx=0x7f0600004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f05f27fc700 1 -- 172.21.15.134:0/2665147618 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06000038a0 con 0x7f06040cd020 2022-11-27T16:17:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f05f27fc700 1 -- 172.21.15.134:0/2665147618 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060001f070 con 0x7f06040cd020 2022-11-27T16:17:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f060c219700 1 -- 172.21.15.134:0/2665147618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06041559d0 con 0x7f06040cd020 2022-11-27T16:17:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f05f27fc700 1 -- 172.21.15.134:0/2665147618 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600003c60 con 0x7f06040cd020 2022-11-27T16:17:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.340+0000 7f060c219700 1 -- 172.21.15.134:0/2665147618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0604152a40 con 0x7f06040cd020 2022-11-27T16:17:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.341+0000 7f060c219700 1 -- 172.21.15.134:0/2665147618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0604073030 con 0x7f06040cd020 2022-11-27T16:17:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.342+0000 7f05f27fc700 1 -- 172.21.15.134:0/2665147618 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f060000f050 con 0x7f06040cd020 2022-11-27T16:17:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.342+0000 7f05f27fc700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f05e4066590 0x7f05e4068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.342+0000 7f05f27fc700 1 -- 172.21.15.134:0/2665147618 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0600021070 con 0x7f06040cd020 2022-11-27T16:17:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.342+0000 7f06097b4700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f05e4066590 0x7f05e4068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.342+0000 7f06097b4700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f05e4066590 0x7f05e4068a40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f06040c8f40 tx=0x7f05f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.346+0000 7f05f27fc700 1 -- 172.21.15.134:0/2665147618 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f06000528f0 con 0x7f06040cd020 2022-11-27T16:17:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.526+0000 7f060c219700 1 -- 172.21.15.134:0/2665147618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f060406c9e0 con 0x7f06040cd020 2022-11-27T16:17:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.527+0000 7f05f27fc700 1 -- 172.21.15.134:0/2665147618 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13789 (secure 0 0 0) 0x7f060002b090 con 0x7f06040cd020 2022-11-27T16:17:18.529 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:17:18.530 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":20,"fsid":"34bf4849-eb1e-4259-a7f0-0be8a82c33c9","created":"2022-11-27T16:16:56.408900+0000","modified":"2022-11-27T16:17:17.812218+0000","last_up_change":"2022-11-27T16:17:07.774037+0000","last_in_change":"2022-11-27T16:16:58.964080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T16:17:09.769201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T16:17:14.856029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f44064b8-6282-45ff-94ab-f3a2eb231dc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":101352},{"type":"v1","addr":"172.21.15.134:6809","nonce":101352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":101352},{"type":"v1","addr":"172.21.15.134:6811","nonce":101352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":101352},{"type":"v1","addr":"172.21.15.134:6815","nonce":101352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":101352},{"type":"v1","addr":"172.21.15.134:6813","nonce":101352}]},"public_addr":"172.21.15.134:6809/101352","cluster_addr":"172.21.15.134:6811/101352","heartbeat_back_addr":"172.21.15.134:6815/101352","heartbeat_front_addr":"172.21.15.134:6813/101352","state":["exists","up"]},{"osd":1,"uuid":"af1b8ac9-0335-4550-aaf8-9f15545625e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":101351},{"type":"v1","addr":"172.21.15.134:6801","nonce":101351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":101351},{"type":"v1","addr":"172.21.15.134:6803","nonce":101351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":101351},{"type":"v1","addr":"172.21.15.134:6807","nonce":101351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":101351},{"type":"v1","addr":"172.21.15.134:6805","nonce":101351}]},"public_addr":"172.21.15.134:6801/101351","cluster_addr":"172.21.15.134:6803/101351","heartbeat_back_addr":"172.21.15.134:6807/101351","heartbeat_front_addr":"172.21.15.134:6805/101351","state":["exists","up"]},{"osd":2,"uuid":"5bbdd443-3ec5-40fb-86af-3e5a6ab1606a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":101353},{"type":"v1","addr":"172.21.15.134:6817","nonce":101353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":101353},{"type":"v1","addr":"172.21.15.134:6819","nonce":101353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":101353},{"type":"v1","addr":"172.21.15.134:6823","nonce":101353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":101353},{"type":"v1","addr":"172.21.15.134:6821","nonce":101353}]},"public_addr":"172.21.15.134:6817/101353","cluster_addr":"172.21.15.134:6819/101353","heartbeat_back_addr":"172.21.15.134:6823/101353","heartbeat_front_addr":"172.21.15.134:6821/101353","state":["exists","up"]},{"osd":3,"uuid":"795fd2fb-0e2e-4419-9b54-ea28ba0ee835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":101371},{"type":"v1","addr":"172.21.15.134:6825","nonce":101371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":101371},{"type":"v1","addr":"172.21.15.134:6827","nonce":101371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":101371},{"type":"v1","addr":"172.21.15.134:6831","nonce":101371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":101371},{"type":"v1","addr":"172.21.15.134:6829","nonce":101371}]},"public_addr":"172.21.15.134:6825/101371","cluster_addr":"172.21.15.134:6827/101371","heartbeat_back_addr":"172.21.15.134:6831/101371","heartbeat_front_addr":"172.21.15.134:6829/101371","state":["exists","up"]},{"osd":4,"uuid":"36005e5e-73a9-4d10-aba3-480165df1594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":101536},{"type":"v1","addr":"172.21.15.191:6817","nonce":101536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":101536},{"type":"v1","addr":"172.21.15.191:6819","nonce":101536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":101536},{"type":"v1","addr":"172.21.15.191:6823","nonce":101536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":101536},{"type":"v1","addr":"172.21.15.191:6821","nonce":101536}]},"public_addr":"172.21.15.191:6817/101536","cluster_addr":"172.21.15.191:6819/101536","heartbeat_back_addr":"172.21.15.191:6823/101536","heartbeat_front_addr":"172.21.15.191:6821/101536","state":["exists","up"]},{"osd":5,"uuid":"6f5b5e55-e69b-4097-a47c-cd7fc0681dbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":101534},{"type":"v1","addr":"172.21.15.191:6801","nonce":101534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":101534},{"type":"v1","addr":"172.21.15.191:6803","nonce":101534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":101534},{"type":"v1","addr":"172.21.15.191:6807","nonce":101534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":101534},{"type":"v1","addr":"172.21.15.191:6805","nonce":101534}]},"public_addr":"172.21.15.191:6801/101534","cluster_addr":"172.21.15.191:6803/101534","heartbeat_back_addr":"172.21.15.191:6807/101534","heartbeat_front_addr":"172.21.15.191:6805/101534","state":["exists","up"]},{"osd":6,"uuid":"074077c9-fc46-4f79-8368-5a96eba6da06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":101545},{"type":"v1","addr":"172.21.15.191:6825","nonce":101545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":101545},{"type":"v1","addr":"172.21.15.191:6827","nonce":101545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":101545},{"type":"v1","addr":"172.21.15.191:6831","nonce":101545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":101545},{"type":"v1","addr":"172.21.15.191:6829","nonce":101545}]},"public_addr":"172.21.15.191:6825/101545","cluster_addr":"172.21.15.191:6827/101545","heartbeat_back_addr":"172.21.15.191:6831/101545","heartbeat_front_addr":"172.21.15.191:6829/101545","state":["exists","up"]},{"osd":7,"uuid":"112889bf-b637-4500-a30d-03e23e8a3267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":101535},{"type":"v1","addr":"172.21.15.191:6809","nonce":101535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":101535},{"type":"v1","addr":"172.21.15.191:6811","nonce":101535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":101535},{"type":"v1","addr":"172.21.15.191:6815","nonce":101535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":101535},{"type":"v1","addr":"172.21.15.191:6813","nonce":101535}]},"public_addr":"172.21.15.191:6809/101535","cluster_addr":"172.21.15.191:6811/101535","heartbeat_back_addr":"172.21.15.191:6815/101535","heartbeat_front_addr":"172.21.15.191:6813/101535","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.141813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:03.885025+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.127132+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.180968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.193639+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.201380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.195434+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:17:04.198016+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:17:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 -- 172.21.15.134:0/2665147618 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f05e4066590 msgr2=0x7f05e4068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f05e4066590 0x7f05e4068a40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f06040c8f40 tx=0x7f05f4009040 comp rx=0 tx=0).stop 2022-11-27T16:17:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 -- 172.21.15.134:0/2665147618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06040cd020 msgr2=0x7f0604157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06040cd020 0x7f0604157210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0600000c00 tx=0x7f0600004750 comp rx=0 tx=0).stop 2022-11-27T16:17:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 -- 172.21.15.134:0/2665147618 shutdown_connections 2022-11-27T16:17:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060415d3a0 0x7f0604152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06040cd020 0x7f0604157210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f05e4066590 0x7f05e4068a40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 --2- 172.21.15.134:0/2665147618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0604158c40 0x7f06041591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 -- 172.21.15.134:0/2665147618 >> 172.21.15.134:0/2665147618 conn(0x7f060406a500 msgr2=0x7f0604153950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 -- 172.21.15.134:0/2665147618 shutdown_connections 2022-11-27T16:17:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.529+0000 7f060c219700 1 -- 172.21.15.134:0/2665147618 wait complete. 2022-11-27T16:17:18.543 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-11-27T16:17:09.769201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T16:17:14.856029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T16:17:18.544 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T16:17:18.667 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.665+0000 7f33a6ade700 1 Processor -- start 2022-11-27T16:17:18.667 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.665+0000 7f33a6ade700 1 -- start start 2022-11-27T16:17:18.670 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a6ade700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f33a015bb10 0x7f33a00693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.671 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a6ade700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f33a00cf8c0 0x7f33a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.671 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a6ade700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a006c050 con 0x7f33a015bee0 2022-11-27T16:17:18.671 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a6ade700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a0069c40 con 0x7f33a00cf8c0 2022-11-27T16:17:18.672 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a6ade700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a0069f20 con 0x7f33a015bb10 2022-11-27T16:17:18.672 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a507b700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f33a00cf8c0 0x7f33a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.672 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f339ffff700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f33a015bee0 0x7f33a006c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56170/0 (socket says 172.21.15.134:56170) 2022-11-27T16:17:18.672 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a507b700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f33a00cf8c0 0x7f33a006d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38102/0 (socket says 172.21.15.134:38102) 2022-11-27T16:17:18.673 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f339ffff700 1 -- 172.21.15.134:0/2607234819 learned_addr learned my addr 172.21.15.134:0/2607234819 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:17:18.673 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a487a700 1 --2- 172.21.15.134:0/2607234819 >> v2:172.21.15.191:3300/0 conn(0x7f33a015bb10 0x7f33a00693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.673 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a507b700 1 -- 172.21.15.134:0/2607234819 >> v2:172.21.15.191:3300/0 conn(0x7f33a015bb10 msgr2=0x7f33a00693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:18.674 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a507b700 1 --2- 172.21.15.134:0/2607234819 >> v2:172.21.15.191:3300/0 conn(0x7f33a015bb10 0x7f33a00693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.674 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a507b700 1 -- 172.21.15.134:0/2607234819 >> v1:172.21.15.134:6789/0 conn(0x7f33a015bee0 legacy=0x7f33a006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.674 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f339f7fe700 1 -- 172.21.15.134:0/2607234819 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573384322 0 0) 0x7f33a006c050 con 0x7f33a015bee0 2022-11-27T16:17:18.674 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a507b700 1 -- 172.21.15.134:0/2607234819 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a006a200 con 0x7f33a00cf8c0 2022-11-27T16:17:18.675 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f33a507b700 1 --2- 172.21.15.134:0/2607234819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a00cf8c0 0x7f33a006d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f33a0072a00 tx=0x7f339800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7deeb0358b5d55f server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:18.675 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f339f7fe700 1 -- 172.21.15.134:0/2607234819 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339800e070 con 0x7f33a00cf8c0 2022-11-27T16:17:18.675 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f339f7fe700 1 -- 172.21.15.134:0/2607234819 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339800bbd0 con 0x7f33a00cf8c0 2022-11-27T16:17:18.675 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.666+0000 7f339f7fe700 1 -- 172.21.15.134:0/2607234819 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398012430 con 0x7f33a00cf8c0 2022-11-27T16:17:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.667+0000 7f33a6ade700 1 -- 172.21.15.134:0/2607234819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a00cf8c0 msgr2=0x7f33a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.667+0000 7f33a6ade700 1 --2- 172.21.15.134:0/2607234819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a00cf8c0 0x7f33a006d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f33a0072a00 tx=0x7f339800b330 comp rx=0 tx=0).stop 2022-11-27T16:17:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.667+0000 7f33a6ade700 1 -- 172.21.15.134:0/2607234819 shutdown_connections 2022-11-27T16:17:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.667+0000 7f33a6ade700 1 --2- 172.21.15.134:0/2607234819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a00cf8c0 0x7f33a006d390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.667+0000 7f33a6ade700 1 --2- 172.21.15.134:0/2607234819 >> v2:172.21.15.191:3300/0 conn(0x7f33a015bb10 0x7f33a00693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.667+0000 7f33a6ade700 1 -- 172.21.15.134:0/2607234819 >> 172.21.15.134:0/2607234819 conn(0x7f33a00b16c0 msgr2=0x7f33a00cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.667+0000 7f33a6ade700 1 -- 172.21.15.134:0/2607234819 shutdown_connections 2022-11-27T16:17:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.667+0000 7f33a6ade700 1 -- 172.21.15.134:0/2607234819 wait complete. 2022-11-27T16:17:18.678 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.668+0000 7f33a6ade700 1 Processor -- start 2022-11-27T16:17:18.678 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.669+0000 7f33a6ade700 1 -- start start 2022-11-27T16:17:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a6ade700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33a00cf8c0 0x7f33a014b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a6ade700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a015bb10 0x7f33a01f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a6ade700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a015bee0 0x7f33a00cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a6ade700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a01591d0 con 0x7f33a015bb10 2022-11-27T16:17:18.684 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a6ade700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a014bec0 con 0x7f33a015bee0 2022-11-27T16:17:18.684 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a6ade700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00ab8d0 con 0x7f33a00cf8c0 2022-11-27T16:17:18.684 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a015bb10 0x7f33a01f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.684 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a015bb10 0x7f33a01f16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38106/0 (socket says 172.21.15.134:38106) 2022-11-27T16:17:18.685 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339ffff700 1 -- 172.21.15.134:0/770772654 learned_addr learned my addr 172.21.15.134:0/770772654 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:18.685 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a507b700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a015bee0 0x7f33a00cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.685 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a487a700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33a00cf8c0 0x7f33a014b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.686 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339ffff700 1 -- 172.21.15.134:0/770772654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33a00cf8c0 msgr2=0x7f33a014b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.686 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339ffff700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33a00cf8c0 0x7f33a014b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.686 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339ffff700 1 -- 172.21.15.134:0/770772654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a015bee0 msgr2=0x7f33a00cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.687 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339ffff700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a015bee0 0x7f33a00cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.687 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339ffff700 1 -- 172.21.15.134:0/770772654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398009a70 con 0x7f33a015bb10 2022-11-27T16:17:18.687 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a507b700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a015bee0 0x7f33a00cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:17:18.687 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a487a700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33a00cf8c0 0x7f33a014b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:17:18.688 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339ffff700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a015bb10 0x7f33a01f16f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f338c00c490 tx=0x7f338c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:18.688 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339d7fa700 1 -- 172.21.15.134:0/770772654 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c004c80 con 0x7f33a015bb10 2022-11-27T16:17:18.688 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339d7fa700 1 -- 172.21.15.134:0/770772654 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c004e00 con 0x7f33a015bb10 2022-11-27T16:17:18.688 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a6ade700 1 -- 172.21.15.134:0/770772654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a01f1c30 con 0x7f33a015bb10 2022-11-27T16:17:18.688 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f339d7fa700 1 -- 172.21.15.134:0/770772654 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c007310 con 0x7f33a015bb10 2022-11-27T16:17:18.689 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.670+0000 7f33a6ade700 1 -- 172.21.15.134:0/770772654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a00abb20 con 0x7f33a015bb10 2022-11-27T16:17:18.689 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.671+0000 7f33a6ade700 1 -- 172.21.15.134:0/770772654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a0065ac0 con 0x7f33a015bb10 2022-11-27T16:17:18.689 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.672+0000 7f339d7fa700 1 -- 172.21.15.134:0/770772654 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f338c01a020 con 0x7f33a015bb10 2022-11-27T16:17:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.672+0000 7f339d7fa700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f3388066590 0x7f3388068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.672+0000 7f339d7fa700 1 -- 172.21.15.134:0/770772654 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f338c082f80 con 0x7f33a015bb10 2022-11-27T16:17:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.672+0000 7f33a487a700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f3388066590 0x7f3388068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.672+0000 7f33a487a700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f3388066590 0x7f3388068a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33a00ca930 tx=0x7f3394008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.676+0000 7f339d7fa700 1 -- 172.21.15.134:0/770772654 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f338c0509e0 con 0x7f33a015bb10 2022-11-27T16:17:18.860 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.858+0000 7f33a6ade700 1 -- 172.21.15.134:0/770772654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33a006c310 con 0x7f33a015bb10 2022-11-27T16:17:18.860 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.858+0000 7f339d7fa700 1 -- 172.21.15.134:0/770772654 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f338c054000 con 0x7f33a015bb10 2022-11-27T16:17:18.861 INFO:teuthology.orchestra.run.smithi134.stdout:pg_num: 1 2022-11-27T16:17:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 -- 172.21.15.134:0/770772654 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f3388066590 msgr2=0x7f3388068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f3388066590 0x7f3388068a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33a00ca930 tx=0x7f3394008040 comp rx=0 tx=0).stop 2022-11-27T16:17:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 -- 172.21.15.134:0/770772654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a015bb10 msgr2=0x7f33a01f16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a015bb10 0x7f33a01f16f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f338c00c490 tx=0x7f338c00c850 comp rx=0 tx=0).stop 2022-11-27T16:17:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 -- 172.21.15.134:0/770772654 shutdown_connections 2022-11-27T16:17:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f3388066590 0x7f3388068a40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a015bee0 0x7f33a00cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33a00cf8c0 0x7f33a014b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 --2- 172.21.15.134:0/770772654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a015bb10 0x7f33a01f16f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 -- 172.21.15.134:0/770772654 >> 172.21.15.134:0/770772654 conn(0x7f33a00b16c0 msgr2=0x7f33a00b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.860+0000 7f33a6ade700 1 -- 172.21.15.134:0/770772654 shutdown_connections 2022-11-27T16:17:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.861+0000 7f33a6ade700 1 -- 172.21.15.134:0/770772654 wait complete. 2022-11-27T16:17:18.874 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T16:17:18.996 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.994+0000 7f7ff12e3700 1 Processor -- start 2022-11-27T16:17:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.994+0000 7f7ff12e3700 1 -- start start 2022-11-27T16:17:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.995+0000 7f7ff12e3700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7fec0b0050 0x7f7fec0693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.995+0000 7f7ff12e3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7fec0b0420 0x7f7fec0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.995+0000 7f7ff12e3700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec070a80 con 0x7f7fec0c8c20 2022-11-27T16:17:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.995+0000 7f7ff12e3700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fec070910 con 0x7f7fec0b0050 2022-11-27T16:17:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.995+0000 7f7ff12e3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fec0699d0 con 0x7f7fec0b0420 2022-11-27T16:17:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.995+0000 7f7feaffd700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7fec0b0050 0x7f7fec0693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.995+0000 7f7feaffd700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7fec0b0050 0x7f7fec0693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38112/0 (socket says 172.21.15.134:38112) 2022-11-27T16:17:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.995+0000 7f7feb7fe700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f7fec0c8c20 0x7f7fec070f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56188/0 (socket says 172.21.15.134:56188) 2022-11-27T16:17:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.995+0000 7f7feaffd700 1 -- 172.21.15.134:0/4274429340 learned_addr learned my addr 172.21.15.134:0/4274429340 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7fea7fc700 1 --2- 172.21.15.134:0/4274429340 >> v2:172.21.15.191:3300/0 conn(0x7f7fec0b0420 0x7f7fec0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7feaffd700 1 -- 172.21.15.134:0/4274429340 >> v2:172.21.15.191:3300/0 conn(0x7f7fec0b0420 msgr2=0x7f7fec0ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7feaffd700 1 --2- 172.21.15.134:0/4274429340 >> v2:172.21.15.191:3300/0 conn(0x7f7fec0b0420 0x7f7fec0ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7feaffd700 1 -- 172.21.15.134:0/4274429340 >> v1:172.21.15.134:6789/0 conn(0x7f7fec0c8c20 legacy=0x7f7fec070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7fe9ffb700 1 -- 172.21.15.134:0/4274429340 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921647035 0 0) 0x7f7fec070a80 con 0x7f7fec0c8c20 2022-11-27T16:17:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7feaffd700 1 -- 172.21.15.134:0/4274429340 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec069cb0 con 0x7f7fec0b0050 2022-11-27T16:17:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7feaffd700 1 --2- 172.21.15.134:0/4274429340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0b0050 0x7f7fec0693f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00098f0 tx=0x7f7fe0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f609136ec3953998 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7fe9ffb700 1 -- 172.21.15.134:0/4274429340 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000b860 con 0x7f7fec0b0050 2022-11-27T16:17:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7fe9ffb700 1 -- 172.21.15.134:0/4274429340 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe000b9c0 con 0x7f7fec0b0050 2022-11-27T16:17:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7fe9ffb700 1 -- 172.21.15.134:0/4274429340 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000bd50 con 0x7f7fec0b0050 2022-11-27T16:17:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7ff12e3700 1 -- 172.21.15.134:0/4274429340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0b0050 msgr2=0x7f7fec0693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.996+0000 7f7ff12e3700 1 --2- 172.21.15.134:0/4274429340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0b0050 0x7f7fec0693f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00098f0 tx=0x7f7fe0009c00 comp rx=0 tx=0).stop 2022-11-27T16:17:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.997+0000 7f7ff12e3700 1 -- 172.21.15.134:0/4274429340 shutdown_connections 2022-11-27T16:17:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.997+0000 7f7ff12e3700 1 --2- 172.21.15.134:0/4274429340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0b0050 0x7f7fec0693f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.997+0000 7f7ff12e3700 1 --2- 172.21.15.134:0/4274429340 >> v2:172.21.15.191:3300/0 conn(0x7f7fec0b0420 0x7f7fec0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.997+0000 7f7ff12e3700 1 -- 172.21.15.134:0/4274429340 >> 172.21.15.134:0/4274429340 conn(0x7f7fec0cb050 msgr2=0x7f7fec0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.997+0000 7f7ff12e3700 1 -- 172.21.15.134:0/4274429340 shutdown_connections 2022-11-27T16:17:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.997+0000 7f7ff12e3700 1 -- 172.21.15.134:0/4274429340 wait complete. 2022-11-27T16:17:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.998+0000 7f7ff12e3700 1 Processor -- start 2022-11-27T16:17:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.998+0000 7f7ff12e3700 1 -- start start 2022-11-27T16:17:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7ff12e3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fec0b0050 0x7f7fec14b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7ff12e3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fec0b0420 0x7f7fec0c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7ff12e3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0c8c20 0x7f7fec0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7ff12e3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec0c4720 con 0x7f7fec0c8c20 2022-11-27T16:17:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7ff12e3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec15e240 con 0x7f7fec0b0420 2022-11-27T16:17:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7ff12e3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec0c4ff0 con 0x7f7fec0b0050 2022-11-27T16:17:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7feb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0c8c20 0x7f7fec0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7feb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0c8c20 0x7f7fec0c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38122/0 (socket says 172.21.15.134:38122) 2022-11-27T16:17:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7feb7fe700 1 -- 172.21.15.134:0/727878172 learned_addr learned my addr 172.21.15.134:0/727878172 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7fea7fc700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fec0b0420 0x7f7fec0c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:18.999+0000 7f7feaffd700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fec0b0050 0x7f7fec14b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7feb7fe700 1 -- 172.21.15.134:0/727878172 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fec0b0050 msgr2=0x7f7fec14b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7feb7fe700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fec0b0050 0x7f7fec14b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7feb7fe700 1 -- 172.21.15.134:0/727878172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fec0b0420 msgr2=0x7f7fec0c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7feb7fe700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fec0b0420 0x7f7fec0c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7feb7fe700 1 -- 172.21.15.134:0/727878172 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe0009550 con 0x7f7fec0c8c20 2022-11-27T16:17:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7feaffd700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fec0b0050 0x7f7fec14b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:17:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7feb7fe700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0c8c20 0x7f7fec0c76b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400cc00 tx=0x7f7fe400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7fd37fe700 1 -- 172.21.15.134:0/727878172 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400aa50 con 0x7f7fec0c8c20 2022-11-27T16:17:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7fd37fe700 1 -- 172.21.15.134:0/727878172 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe400abd0 con 0x7f7fec0c8c20 2022-11-27T16:17:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7ff12e3700 1 -- 172.21.15.134:0/727878172 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fec15ddb0 con 0x7f7fec0c8c20 2022-11-27T16:17:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7fd37fe700 1 -- 172.21.15.134:0/727878172 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe40070d0 con 0x7f7fec0c8c20 2022-11-27T16:17:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.000+0000 7f7ff12e3700 1 -- 172.21.15.134:0/727878172 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec15d760 con 0x7f7fec0c8c20 2022-11-27T16:17:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.001+0000 7f7ff12e3700 1 -- 172.21.15.134:0/727878172 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fec15d5d0 con 0x7f7fec0c8c20 2022-11-27T16:17:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.001+0000 7f7fd37fe700 1 -- 172.21.15.134:0/727878172 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f7fe401a030 con 0x7f7fec0c8c20 2022-11-27T16:17:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.002+0000 7f7fd37fe700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f7fcc066590 0x7f7fcc068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.002+0000 7f7fd37fe700 1 -- 172.21.15.134:0/727878172 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7fe4083060 con 0x7f7fec0c8c20 2022-11-27T16:17:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.002+0000 7f7feaffd700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f7fcc066590 0x7f7fcc068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.002+0000 7f7feaffd700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f7fcc066590 0x7f7fcc068a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0698a0 tx=0x7f7fe0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.006+0000 7f7fd37fe700 1 -- 172.21.15.134:0/727878172 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f7fe4050ac0 con 0x7f7fec0c8c20 2022-11-27T16:17:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.184+0000 7f7ff12e3700 1 -- 172.21.15.134:0/727878172 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7fec15df40 con 0x7f7fec0c8c20 2022-11-27T16:17:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.184+0000 7f7fd37fe700 1 -- 172.21.15.134:0/727878172 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7fe4018070 con 0x7f7fec0c8c20 2022-11-27T16:17:19.187 INFO:teuthology.orchestra.run.smithi134.stdout:pg_num: 8 2022-11-27T16:17:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 -- 172.21.15.134:0/727878172 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f7fcc066590 msgr2=0x7f7fcc068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f7fcc066590 0x7f7fcc068a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0698a0 tx=0x7f7fe0008030 comp rx=0 tx=0).stop 2022-11-27T16:17:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 -- 172.21.15.134:0/727878172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0c8c20 msgr2=0x7f7fec0c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0c8c20 0x7f7fec0c76b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400cc00 tx=0x7f7fe400cf10 comp rx=0 tx=0).stop 2022-11-27T16:17:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 -- 172.21.15.134:0/727878172 shutdown_connections 2022-11-27T16:17:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fec0c8c20 0x7f7fec0c76b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fec0b0050 0x7f7fec14b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f7fcc066590 0x7f7fcc068a40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 --2- 172.21.15.134:0/727878172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fec0b0420 0x7f7fec0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 -- 172.21.15.134:0/727878172 >> 172.21.15.134:0/727878172 conn(0x7f7fec0cb050 msgr2=0x7f7fec0ca220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 -- 172.21.15.134:0/727878172 shutdown_connections 2022-11-27T16:17:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.186+0000 7f7ff12e3700 1 -- 172.21.15.134:0/727878172 wait complete. 2022-11-27T16:17:19.200 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-11-27T16:17:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.318+0000 7fc74b83f700 1 Processor -- start 2022-11-27T16:17:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.318+0000 7fc74b83f700 1 -- start start 2022-11-27T16:17:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.319+0000 7fc74b83f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc7440cd020 0x7fc7440cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.319+0000 7fc74b83f700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc744070070 con 0x7fc744158c40 2022-11-27T16:17:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.319+0000 7fc74b83f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7440cd9a0 con 0x7fc74415d3a0 2022-11-27T16:17:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.319+0000 7fc74b83f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74406c780 con 0x7fc7440cd020 2022-11-27T16:17:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.319+0000 7fc7495db700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc744158c40 0x7fc744070da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56198/0 (socket says 172.21.15.134:56198) 2022-11-27T16:17:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.319+0000 7fc7495db700 1 -- 172.21.15.134:0/3199830148 learned_addr learned my addr 172.21.15.134:0/3199830148 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:17:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.319+0000 7fc749ddc700 1 --2- 172.21.15.134:0/3199830148 >> v2:172.21.15.191:3300/0 conn(0x7fc7440cd020 0x7fc7440cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.319+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192145710 0 0) 0x7fc744070070 con 0x7fc744158c40 2022-11-27T16:17:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.319+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc72c0035b0 con 0x7fc744158c40 2022-11-27T16:17:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 559158279 0 0) 0x7fc72c0035b0 con 0x7fc744158c40 2022-11-27T16:17:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 >> v2:172.21.15.191:3300/0 conn(0x7fc7440cd020 msgr2=0x7fc7440cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 --2- 172.21.15.134:0/3199830148 >> v2:172.21.15.191:3300/0 conn(0x7fc7440cd020 0x7fc7440cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 >> v1:172.21.15.191:6789/0 conn(0x7fc74415d3a0 legacy=0x7fc74406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc744159010 con 0x7fc744158c40 2022-11-27T16:17:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2726658286 0 0) 0x7fc734001670 con 0x7fc744158c40 2022-11-27T16:17:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 >> v1:172.21.15.134:6789/0 conn(0x7fc744158c40 legacy=0x7fc744070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc72c003a00 0x7fc72c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc72c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc72c005200 0x7fc72c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74406c780 con 0x7fc72c004850 2022-11-27T16:17:19.341 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c00e290 con 0x7fc72c003a00 2022-11-27T16:17:19.341 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c00c0a0 con 0x7fc72c005200 2022-11-27T16:17:19.341 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc748dda700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc72c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.341 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc7495db700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc72c003a00 0x7fc72c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.341 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc749ddc700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc72c005200 0x7fc72c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.342 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc748dda700 1 -- 172.21.15.134:0/3199830148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc72c005200 msgr2=0x7fc72c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:19.342 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc748dda700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc72c005200 0x7fc72c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.342 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc748dda700 1 -- 172.21.15.134:0/3199830148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc72c003a00 msgr2=0x7fc72c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:19.342 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc748dda700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc72c003a00 0x7fc72c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.343 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc748dda700 1 -- 172.21.15.134:0/3199830148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c0035b0 con 0x7fc72c004850 2022-11-27T16:17:19.343 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc749ddc700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc72c005200 0x7fc72c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:17:19.343 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc749ddc700 1 -- 172.21.15.134:0/3199830148 reap_dead start 2022-11-27T16:17:19.343 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc748dda700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc72c004cc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc740012e30 tx=0x7fc72c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a22913c7e581fc server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:19.344 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc740017070 con 0x7fc72c004850 2022-11-27T16:17:19.344 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc740001ad0 con 0x7fc72c004850 2022-11-27T16:17:19.344 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.320+0000 7fc733fff700 1 -- 172.21.15.134:0/3199830148 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7400050d0 con 0x7fc72c004850 2022-11-27T16:17:19.344 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.321+0000 7fc74b83f700 1 -- 172.21.15.134:0/3199830148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 msgr2=0x7fc72c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.344 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.321+0000 7fc74b83f700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc72c004cc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc740012e30 tx=0x7fc72c005880 comp rx=0 tx=0).stop 2022-11-27T16:17:19.345 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.321+0000 7fc74b83f700 1 -- 172.21.15.134:0/3199830148 shutdown_connections 2022-11-27T16:17:19.345 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.321+0000 7fc74b83f700 1 --2- 172.21.15.134:0/3199830148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc72c004cc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.345 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.321+0000 7fc74b83f700 1 -- 172.21.15.134:0/3199830148 >> 172.21.15.134:0/3199830148 conn(0x7fc74406a500 msgr2=0x7fc74406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:19.345 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.321+0000 7fc74b83f700 1 -- 172.21.15.134:0/3199830148 shutdown_connections 2022-11-27T16:17:19.346 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.321+0000 7fc74b83f700 1 -- 172.21.15.134:0/3199830148 wait complete. 2022-11-27T16:17:19.346 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.322+0000 7fc74b83f700 1 Processor -- start 2022-11-27T16:17:19.346 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.322+0000 7fc74b83f700 1 -- start start 2022-11-27T16:17:19.346 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc74b83f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc744069940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc74b83f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7440b3f20 0x7fc744152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc74b83f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7441f00a0 0x7fc7441534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc74b83f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7441440a0 con 0x7fc72c004850 2022-11-27T16:17:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc74b83f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440b4390 con 0x7fc7441f00a0 2022-11-27T16:17:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc74b83f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440b4670 con 0x7fc7440b3f20 2022-11-27T16:17:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc7495db700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc744069940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc7495db700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc744069940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38138/0 (socket says 172.21.15.134:38138) 2022-11-27T16:17:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc7495db700 1 -- 172.21.15.134:0/1464809111 learned_addr learned my addr 172.21.15.134:0/1464809111 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc749ddc700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7441f00a0 0x7fc7441534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc748dda700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7440b3f20 0x7fc744152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc7495db700 1 -- 172.21.15.134:0/1464809111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7440b3f20 msgr2=0x7fc744152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc7495db700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7440b3f20 0x7fc744152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc7495db700 1 -- 172.21.15.134:0/1464809111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7441f00a0 msgr2=0x7fc7441534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc7495db700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7441f00a0 0x7fc7441534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc7495db700 1 -- 172.21.15.134:0/1464809111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440b4950 con 0x7fc72c004850 2022-11-27T16:17:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.323+0000 7fc749ddc700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7441f00a0 0x7fc7441534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:17:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.324+0000 7fc7495db700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc744069940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc734007290 tx=0x7fc734007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.324+0000 7fc731ffb700 1 -- 172.21.15.134:0/1464809111 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7340014c0 con 0x7fc72c004850 2022-11-27T16:17:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.324+0000 7fc731ffb700 1 -- 172.21.15.134:0/1464809111 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73400f850 con 0x7fc72c004850 2022-11-27T16:17:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.324+0000 7fc731ffb700 1 -- 172.21.15.134:0/1464809111 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73400fc00 con 0x7fc72c004850 2022-11-27T16:17:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.324+0000 7fc74b83f700 1 -- 172.21.15.134:0/1464809111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74413dd70 con 0x7fc72c004850 2022-11-27T16:17:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.324+0000 7fc74b83f700 1 -- 172.21.15.134:0/1464809111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc744069e80 con 0x7fc72c004850 2022-11-27T16:17:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.325+0000 7fc74b83f700 1 -- 172.21.15.134:0/1464809111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74413dfb0 con 0x7fc72c004850 2022-11-27T16:17:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.325+0000 7fc731ffb700 1 -- 172.21.15.134:0/1464809111 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fc73400fd60 con 0x7fc72c004850 2022-11-27T16:17:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.325+0000 7fc731ffb700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc724066590 0x7fc724068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.326+0000 7fc748dda700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc724066590 0x7fc724068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.326+0000 7fc731ffb700 1 -- 172.21.15.134:0/1464809111 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc734083600 con 0x7fc72c004850 2022-11-27T16:17:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.326+0000 7fc748dda700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc724066590 0x7fc724068a40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc740012e30 tx=0x7fc740018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.330+0000 7fc731ffb700 1 -- 172.21.15.134:0/1464809111 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fc734051060 con 0x7fc72c004850 2022-11-27T16:17:19.513 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.511+0000 7fc74b83f700 1 -- 172.21.15.134:0/1464809111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc7440707e0 con 0x7fc72c004850 2022-11-27T16:17:19.514 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.512+0000 7fc731ffb700 1 -- 172.21.15.134:0/1464809111 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc734054680 con 0x7fc72c004850 2022-11-27T16:17:19.514 INFO:teuthology.orchestra.run.smithi134.stdout: 2022-11-27T16:17:19.514 INFO:teuthology.orchestra.run.smithi134.stdout:[ 2022-11-27T16:17:19.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2022-11-27T16:17:19.515 INFO:teuthology.orchestra.run.smithi134.stdout: "poolnum": 1, 2022-11-27T16:17:19.515 INFO:teuthology.orchestra.run.smithi134.stdout: "poolname": ".mgr" 2022-11-27T16:17:19.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2022-11-27T16:17:19.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2022-11-27T16:17:19.516 INFO:teuthology.orchestra.run.smithi134.stdout: "poolnum": 2, 2022-11-27T16:17:19.516 INFO:teuthology.orchestra.run.smithi134.stdout: "poolname": "rbd" 2022-11-27T16:17:19.516 INFO:teuthology.orchestra.run.smithi134.stdout: } 2022-11-27T16:17:19.517 INFO:teuthology.orchestra.run.smithi134.stdout:] 2022-11-27T16:17:19.518 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 -- 172.21.15.134:0/1464809111 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc724066590 msgr2=0x7fc724068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.518 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc724066590 0x7fc724068a40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc740012e30 tx=0x7fc740018000 comp rx=0 tx=0).stop 2022-11-27T16:17:19.518 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 -- 172.21.15.134:0/1464809111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 msgr2=0x7fc744069940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.519 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc744069940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc734007290 tx=0x7fc734007650 comp rx=0 tx=0).stop 2022-11-27T16:17:19.519 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 -- 172.21.15.134:0/1464809111 shutdown_connections 2022-11-27T16:17:19.519 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7fc724066590 0x7fc724068a40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.519 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7441f00a0 0x7fc7441534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.520 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc72c004850 0x7fc744069940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.520 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 --2- 172.21.15.134:0/1464809111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7440b3f20 0x7fc744152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.520 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 -- 172.21.15.134:0/1464809111 >> 172.21.15.134:0/1464809111 conn(0x7fc74406a500 msgr2=0x7fc7440ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:19.520 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 -- 172.21.15.134:0/1464809111 shutdown_connections 2022-11-27T16:17:19.521 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.514+0000 7fc74b83f700 1 -- 172.21.15.134:0/1464809111 wait complete. 2022-11-27T16:17:19.527 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-11-27T16:17:19.528 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2022-11-27T16:17:19.651 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.649+0000 7f5772f14700 1 Processor -- start 2022-11-27T16:17:19.651 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.649+0000 7f5772f14700 1 -- start start 2022-11-27T16:17:19.654 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.650+0000 7f5772f14700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f576c0cf8c0 0x7f576c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.655 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.650+0000 7f5772f14700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576c06c050 con 0x7f576c15bb10 2022-11-27T16:17:19.655 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.650+0000 7f5772f14700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576c069c40 con 0x7f576c15bee0 2022-11-27T16:17:19.655 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.650+0000 7f5772f14700 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c069f20 con 0x7f576c0cf8c0 2022-11-27T16:17:19.655 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.650+0000 7f57714b1700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f576c0cf8c0 0x7f576c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.656 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.650+0000 7f5770cb0700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f576c15bb10 0x7f576c0693f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56218/0 (socket says 172.21.15.134:56218) 2022-11-27T16:17:19.656 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.650+0000 7f57714b1700 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f576c0cf8c0 0x7f576c06d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38150/0 (socket says 172.21.15.134:38150) 2022-11-27T16:17:19.656 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.650+0000 7f5770cb0700 1 -- 172.21.15.134:0/589015300 learned_addr learned my addr 172.21.15.134:0/589015300 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:17:19.656 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f57714b1700 1 -- 172.21.15.134:0/589015300 >> v1:172.21.15.191:6789/0 conn(0x7f576c15bee0 legacy=0x7f576c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.657 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f57714b1700 1 -- 172.21.15.134:0/589015300 >> v1:172.21.15.134:6789/0 conn(0x7f576c15bb10 legacy=0x7f576c0693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.657 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f57714b1700 1 -- 172.21.15.134:0/589015300 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c06a200 con 0x7f576c0cf8c0 2022-11-27T16:17:19.657 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f57714b1700 1 --2- 172.21.15.134:0/589015300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 0x7f576c06d390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5764009d90 tx=0x7f576400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe9be5715f9973f server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:19.657 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f576b7fe700 1 -- 172.21.15.134:0/589015300 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576400e070 con 0x7f576c0cf8c0 2022-11-27T16:17:19.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f576b7fe700 1 -- 172.21.15.134:0/589015300 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576400bbd0 con 0x7f576c0cf8c0 2022-11-27T16:17:19.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f576b7fe700 1 -- 172.21.15.134:0/589015300 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764012430 con 0x7f576c0cf8c0 2022-11-27T16:17:19.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f5772f14700 1 -- 172.21.15.134:0/589015300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 msgr2=0x7f576c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f5772f14700 1 --2- 172.21.15.134:0/589015300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 0x7f576c06d390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5764009d90 tx=0x7f576400b330 comp rx=0 tx=0).stop 2022-11-27T16:17:19.658 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f5772f14700 1 -- 172.21.15.134:0/589015300 shutdown_connections 2022-11-27T16:17:19.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f5772f14700 1 --2- 172.21.15.134:0/589015300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 0x7f576c06d390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f5772f14700 1 -- 172.21.15.134:0/589015300 >> 172.21.15.134:0/589015300 conn(0x7f576c0b16c0 msgr2=0x7f576c0cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:19.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f5772f14700 1 -- 172.21.15.134:0/589015300 shutdown_connections 2022-11-27T16:17:19.659 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.651+0000 7f5772f14700 1 -- 172.21.15.134:0/589015300 wait complete. 2022-11-27T16:17:19.660 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.653+0000 7f5772f14700 1 Processor -- start 2022-11-27T16:17:19.660 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.653+0000 7f5772f14700 1 -- start start 2022-11-27T16:17:19.663 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.653+0000 7f5772f14700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 0x7f576c1f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.663 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.653+0000 7f5772f14700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f576c15bb10 0x7f576c0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.663 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.653+0000 7f5772f14700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f576c15bee0 0x7f576c0ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.664 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.653+0000 7f5772f14700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c1f1db0 con 0x7f576c0cf8c0 2022-11-27T16:17:19.664 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.653+0000 7f5772f14700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0b07e0 con 0x7f576c15bb10 2022-11-27T16:17:19.664 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.653+0000 7f5772f14700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0afa20 con 0x7f576c15bee0 2022-11-27T16:17:19.664 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5770cb0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 0x7f576c1f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.664 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5770cb0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 0x7f576c1f1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38152/0 (socket says 172.21.15.134:38152) 2022-11-27T16:17:19.665 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5770cb0700 1 -- 172.21.15.134:0/2821923373 learned_addr learned my addr 172.21.15.134:0/2821923373 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:19.665 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f576bfff700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f576c15bb10 0x7f576c0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.665 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f57714b1700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f576c15bee0 0x7f576c0ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.665 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5770cb0700 1 -- 172.21.15.134:0/2821923373 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f576c15bee0 msgr2=0x7f576c0ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.666 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5770cb0700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f576c15bee0 0x7f576c0ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.666 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5770cb0700 1 -- 172.21.15.134:0/2821923373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f576c15bb10 msgr2=0x7f576c0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:19.666 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5770cb0700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f576c15bb10 0x7f576c0ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:19.666 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5770cb0700 1 -- 172.21.15.134:0/2821923373 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5764009a70 con 0x7f576c0cf8c0 2022-11-27T16:17:19.667 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5770cb0700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 0x7f576c1f1870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f576000c520 tx=0x7f576000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:19.667 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f57697fa700 1 -- 172.21.15.134:0/2821923373 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760005d70 con 0x7f576c0cf8c0 2022-11-27T16:17:19.667 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f57697fa700 1 -- 172.21.15.134:0/2821923373 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57600070d0 con 0x7f576c0cf8c0 2022-11-27T16:17:19.667 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f57697fa700 1 -- 172.21.15.134:0/2821923373 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760007440 con 0x7f576c0cf8c0 2022-11-27T16:17:19.668 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5772f14700 1 -- 172.21.15.134:0/2821923373 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c0afd00 con 0x7f576c0cf8c0 2022-11-27T16:17:19.668 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.654+0000 7f5772f14700 1 -- 172.21.15.134:0/2821923373 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c0b02c0 con 0x7f576c0cf8c0 2022-11-27T16:17:19.668 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.655+0000 7f5772f14700 1 -- 172.21.15.134:0/2821923373 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576c065ac0 con 0x7f576c0cf8c0 2022-11-27T16:17:19.668 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.656+0000 7f57697fa700 1 -- 172.21.15.134:0/2821923373 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f5760019030 con 0x7f576c0cf8c0 2022-11-27T16:17:19.668 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.656+0000 7f57697fa700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f5754066590 0x7f5754068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:19.669 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.656+0000 7f57697fa700 1 -- 172.21.15.134:0/2821923373 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f57600825c0 con 0x7f576c0cf8c0 2022-11-27T16:17:19.669 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.656+0000 7f576bfff700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f5754066590 0x7f5754068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:19.669 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.656+0000 7f576bfff700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f5754066590 0x7f5754068a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f576c0af2d0 tx=0x7f5758009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:19.669 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.660+0000 7f57697fa700 1 -- 172.21.15.134:0/2821923373 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f576004abd0 con 0x7f576c0cf8c0 2022-11-27T16:17:19.846 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:19.844+0000 7f5772f14700 1 -- 172.21.15.134:0/2821923373 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f576c073030 con 0x7f576c0cf8c0 2022-11-27T16:17:19.999 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:17:20.002 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:17:20.832 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.830+0000 7f57697fa700 1 -- 172.21.15.134:0/2821923373 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7f57600170b0 con 0x7f576c0cf8c0 2022-11-27T16:17:20.837 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.834+0000 7f5772f14700 1 -- 172.21.15.134:0/2821923373 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f5754066590 msgr2=0x7f5754068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:20.837 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.834+0000 7f5772f14700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f5754066590 0x7f5754068a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f576c0af2d0 tx=0x7f5758009040 comp rx=0 tx=0).stop 2022-11-27T16:17:20.838 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.834+0000 7f5772f14700 1 -- 172.21.15.134:0/2821923373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 msgr2=0x7f576c1f1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:20.838 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.834+0000 7f5772f14700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 0x7f576c1f1870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f576000c520 tx=0x7f576000c830 comp rx=0 tx=0).stop 2022-11-27T16:17:20.838 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.835+0000 7f5772f14700 1 -- 172.21.15.134:0/2821923373 shutdown_connections 2022-11-27T16:17:20.838 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.835+0000 7f5772f14700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f576c15bee0 0x7f576c0ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:20.839 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.835+0000 7f5772f14700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f576c0cf8c0 0x7f576c1f1870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:20.839 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.835+0000 7f5772f14700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.134:6832/101079,v1:172.21.15.134:6833/101079] conn(0x7f5754066590 0x7f5754068a40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:20.839 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.835+0000 7f5772f14700 1 --2- 172.21.15.134:0/2821923373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f576c15bb10 0x7f576c0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:20.839 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.835+0000 7f5772f14700 1 -- 172.21.15.134:0/2821923373 >> 172.21.15.134:0/2821923373 conn(0x7f576c0b16c0 msgr2=0x7f576c06d390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:20.840 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.835+0000 7f5772f14700 1 -- 172.21.15.134:0/2821923373 shutdown_connections 2022-11-27T16:17:20.840 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.835+0000 7f5772f14700 1 -- 172.21.15.134:0/2821923373 wait complete. 2022-11-27T16:17:20.840 INFO:teuthology.orchestra.run.smithi134.stderr:pool 'cephfs_metadata' created 2022-11-27T16:17:20.853 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2022-11-27T16:17:20.996 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.985+0000 7f4d46405700 1 Processor -- start 2022-11-27T16:17:20.997 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.985+0000 7f4d46405700 1 -- start start 2022-11-27T16:17:20.997 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.986+0000 7f4d46405700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4d4006ff80 0x7f4d40070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:20.997 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.986+0000 7f4d46405700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400ce510 0x7f4d4006c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:20.997 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.986+0000 7f4d46405700 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d40070890 con 0x7f4d400c4620 2022-11-27T16:17:20.998 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.986+0000 7f4d46405700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d400c4b00 con 0x7f4d4006ff80 2022-11-27T16:17:20.998 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.986+0000 7f4d46405700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400cda30 con 0x7f4d400ce510 2022-11-27T16:17:20.998 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.986+0000 7f4d3f7fe700 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f4d400c4620 0x7f4d400c3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56232/0 (socket says 172.21.15.134:56232) 2022-11-27T16:17:20.998 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.986+0000 7f4d3f7fe700 1 -- 172.21.15.134:0/995674659 learned_addr learned my addr 172.21.15.134:0/995674659 (peer_addr_for_me v1:172.21.15.134:0/0) 2022-11-27T16:17:20.998 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924480664 0 0) 0x7f4d40070890 con 0x7f4d400c4620 2022-11-27T16:17:20.999 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d2c0035b0 con 0x7f4d400c4620 2022-11-27T16:17:20.999 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3ffff700 1 --2- 172.21.15.134:0/995674659 >> v2:172.21.15.191:3300/0 conn(0x7f4d4006ff80 0x7f4d40070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:20.999 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d449a2700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400ce510 0x7f4d4006c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:20.999 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3550556058 0 0) 0x7f4d2c0035b0 con 0x7f4d400c4620 2022-11-27T16:17:21.000 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400ce510 msgr2=0x7f4d4006c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:21.000 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400ce510 0x7f4d4006c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:21.000 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 >> v2:172.21.15.191:3300/0 conn(0x7f4d4006ff80 msgr2=0x7f4d40070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:21.000 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 --2- 172.21.15.134:0/995674659 >> v2:172.21.15.191:3300/0 conn(0x7f4d4006ff80 0x7f4d40070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:21.001 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d400cdd10 con 0x7f4d400c4620 2022-11-27T16:17:21.001 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2726658286 0 0) 0x7f4d30001670 con 0x7f4d400c4620 2022-11-27T16:17:21.001 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d449a2700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400ce510 0x7f4d4006c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:17:21.001 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 >> v1:172.21.15.134:6789/0 conn(0x7f4d400c4620 legacy=0x7f4d400c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:21.001 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d2c003a00 0x7f4d2c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:21.002 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2c004850 0x7f4d2c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:21.002 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400ce510 0x7f4d2c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:21.002 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400c4b00 con 0x7f4d2c004850 2022-11-27T16:17:21.003 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400cda30 con 0x7f4d2c003a00 2022-11-27T16:17:21.003 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c00bbc0 con 0x7f4d400ce510 2022-11-27T16:17:21.003 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3ffff700 1 --2- 172.21.15.134:0/995674659 >> v2:172.21.15.191:3300/0 conn(0x7f4d4006ff80 0x7f4d40070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:17:21.003 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3f7fe700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2c004850 0x7f4d2c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:21.004 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d449a2700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400ce510 0x7f4d2c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:21.004 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3ffff700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d2c003a00 0x7f4d2c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:21.004 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3f7fe700 1 -- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400ce510 msgr2=0x7f4d2c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:21.004 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3f7fe700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400ce510 0x7f4d2c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:21.004 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3f7fe700 1 -- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d2c003a00 msgr2=0x7f4d2c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:17:21.005 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3f7fe700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d2c003a00 0x7f4d2c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:21.005 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3f7fe700 1 -- 172.21.15.134:0/995674659 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c0035b0 con 0x7f4d2c004850 2022-11-27T16:17:21.006 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.987+0000 7f4d3f7fe700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2c004850 0x7f4d2c004cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000c2f0 tx=0x7f4d2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c79966db35767f server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:17:21.006 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.988+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000ce30 con 0x7f4d2c004850 2022-11-27T16:17:21.006 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.988+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d30005960 con 0x7f4d2c004850 2022-11-27T16:17:21.006 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.988+0000 7f4d3effd700 1 -- 172.21.15.134:0/995674659 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30005d10 con 0x7f4d2c004850 2022-11-27T16:17:21.007 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.988+0000 7f4d46405700 1 -- 172.21.15.134:0/995674659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2c004850 msgr2=0x7f4d2c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:21.007 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.988+0000 7f4d46405700 1 --2- 172.21.15.134:0/995674659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2c004850 0x7f4d2c004cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000c2f0 tx=0x7f4d2c005880 comp rx=0 tx=0).stop 2022-11-27T16:17:21.007 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.988+0000 7f4d449a2700 1 -- 172.21.15.134:0/995674659 reap_dead start 2022-11-27T16:17:21.007 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.988+0000 7f4d46405700 1 -- 172.21.15.134:0/995674659 shutdown_connections 2022-11-27T16:17:21.008 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.988+0000 7f4d46405700 1 -- 172.21.15.134:0/995674659 >> 172.21.15.134:0/995674659 conn(0x7f4d400b16c0 msgr2=0x7f4d400cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:17:21.008 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.988+0000 7f4d46405700 1 -- 172.21.15.134:0/995674659 shutdown_connections 2022-11-27T16:17:21.008 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.988+0000 7f4d46405700 1 -- 172.21.15.134:0/995674659 wait complete. 2022-11-27T16:17:21.008 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.989+0000 7f4d46405700 1 Processor -- start 2022-11-27T16:17:21.008 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.990+0000 7f4d46405700 1 -- start start 2022-11-27T16:17:21.009 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.990+0000 7f4d46405700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2c004850 0x7f4d401f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:21.009 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.990+0000 7f4d46405700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400cab90 0x7f4d401454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:21.009 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.990+0000 7f4d46405700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d40157b30 0x7f4d400c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:17:21.010 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.990+0000 7f4d46405700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d401f5490 con 0x7f4d2c004850 2022-11-27T16:17:21.010 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.990+0000 7f4d46405700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40156a90 con 0x7f4d40157b30 2022-11-27T16:17:21.010 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.990+0000 7f4d46405700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40156d70 con 0x7f4d400cab90 2022-11-27T16:17:21.010 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.990+0000 7f4d3ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2c004850 0x7f4d401f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:21.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.990+0000 7f4d3ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d2c004850 0x7f4d401f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38172/0 (socket says 172.21.15.134:38172) 2022-11-27T16:17:21.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.990+0000 7f4d3ffff700 1 -- 172.21.15.134:0/2536274854 learned_addr learned my addr 172.21.15.134:0/2536274854 (peer_addr_for_me v2:172.21.15.134:0/0) 2022-11-27T16:17:21.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.991+0000 7f4d449a2700 1 --2- 172.21.15.134:0/2536274854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d40157b30 0x7f4d400c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:21.011 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.991+0000 7f4d3f7fe700 1 --2- 172.21.15.134:0/2536274854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400cab90 0x7f4d401454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:17:21.012 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.991+0000 7f4d3ffff700 1 -- 172.21.15.134:0/2536274854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400cab90 msgr2=0x7f4d401454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:21.012 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.991+0000 7f4d3ffff700 1 --2- 172.21.15.134:0/2536274854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d400cab90 0x7f4d401454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:21.012 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.991+0000 7f4d3ffff700 1 -- 172.21.15.134:0/2536274854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d40157b30 msgr2=0x7f4d400c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:17:21.013 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.991+0000 7f4d3ffff700 1 --2- 172.21.15.134:0/2536274854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d40157b30 0x7f4d400c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:17:21.013 INFO:teuthology.orchestra.run.smithi134.stderr:2022-11-27T16:17:20.991+0000 7f4d3ffff700 1 -- 172.21.15.134:0/2536274854 --> [v2:172.21.15.134:3300/0,v1:172.21.1