2023-03-24T22:11:15.635 INFO:root:teuthology version: 0.0.1.dev1+ge372916 2023-03-24T22:11:15.637 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} --archive /home/teuthworker/archive/pdonnell-2023-03-24_22:00:37-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219596 --name pdonnell-2023-03-24_22:00:37-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi --verbose -- /home/teuthworker/archive/pdonnell-2023-03-24_22:00:37-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219596/orig.config.yaml 2023-03-24T22:11:15.682 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-03-24T22:11:15.769 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-03-24_22:00:37-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219596 branch: wip-pdonnell-testing-20230323.162417 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} email: pdonnell@redhat.com first_in_suite: false job_id: '7219596' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-03-24_22:00:37-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-pdonnell-testing-20230323.162417 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - inconsistent rstat on inode - Error recovering journal sha1: c19e8f2a497e95f75abc43ec14bc6095e30dd2cb ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c19e8f2a497e95f75abc43ec14bc6095e30dd2cb selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20230323.162417 sha1: 568504667cd0ede116ce6b46562b9a2881100488 owner: scheduled_pdonnell@teuthology priority: 55 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: c19e8f2a497e95f75abc43ec14bc6095e30dd2cb sleep_before_teardown: 0 suite: fs suite_branch: wip-pdonnell-testing-20230323.162417 suite_path: /home/teuthworker/src/github.com_batrick_ceph_568504667cd0ede116ce6b46562b9a2881100488/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: 568504667cd0ede116ce6b46562b9a2881100488 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvUEChdBPIgigUGWvKnPofuW3lrWraE6RMOEel3fhd2Y92ZHtwvLl6znQDLuVCrofIy9kO98KKu4Ojz5LCBYxusXN6MRCvfLhzCWR15cq3DS2YXK5uYqXC9BFF0OD/r/AswpxXleqzCRi7yCsV1MfAveTDT+f5aw7QcoI6Kh3n6sYU2ZYJPtl2HGvwZO36vi+eZMEeGhkWoqKDD56XHrbgZdGv9jg9qB1IJ7sqop8a+l/rQQdhS3ex2FiXyYuAlyQG98oSLa0Gj134E5vB1io1RkK4ZUeh0q6Hurzd9S9whj6i1rAcVff7VRza6EQ+IlS1cZEhX5UZxVj1pAdbIAWBjuN3E+w6mceL+zyjbu9CpJsySEdxZjTVf0v95ZbK+F3R79RzBYEU5h05OD8PGkn8vk91bHumRqTFkuGq+IJolDUB2mUZkEN2eaI51+MEqpMKhaxiqrk4UuncyshnVwxZ/RgxuIlrNkJKE8Y97O75VhapsA7R5OdEhx1IuE3E3Nc= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMDwhBowsBXOFxn/w06cj4LLga3mXdGLyNAtME6AQlxc92sSuKpqEFuftyXyhctHuPs9QO+N8Ul6IaQBaIPGK4U= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_data_scan teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: parallel-gzip teuthology_sha1: fe9b6e52013911547168fef4a8506325b6c76139 timestamp: 2023-03-24_22:00:37 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4538 2023-03-24T22:11:15.770 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_568504667cd0ede116ce6b46562b9a2881100488/qa; will attempt to use it 2023-03-24T22:11:15.771 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_568504667cd0ede116ce6b46562b9a2881100488/qa/tasks 2023-03-24T22:11:15.771 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-03-24T22:11:15.773 INFO:teuthology.task.internal:Checking packages... 2023-03-24T22:11:15.802 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb' 2023-03-24T22:11:15.803 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-03-24T22:11:15.803 INFO:teuthology.packaging:ref: None 2023-03-24T22:11:15.803 INFO:teuthology.packaging:tag: None 2023-03-24T22:11:15.804 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20230323.162417 2023-03-24T22:11:15.804 INFO:teuthology.packaging:sha1: c19e8f2a497e95f75abc43ec14bc6095e30dd2cb 2023-03-24T22:11:15.804 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20230323.162417 2023-03-24T22:11:15.952 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3124.gc19e8f2a 2023-03-24T22:11:15.952 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-03-24T22:11:15.960 INFO:teuthology.task.internal:no buildpackages task found 2023-03-24T22:11:15.960 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-03-24T22:11:15.981 INFO:teuthology.task.internal:Saving configuration 2023-03-24T22:11:15.997 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-03-24T22:11:16.003 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-24T22:11:16.034 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-03-24_22:00:37-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219596', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-24 22:05:59.288555', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNNr0sL3IynOp94qcvY5X0D7NpustRsUFHhe8Y9Rl6dhAG3ZgCk493em4Nf0oqOJSxhm1mAIyA/AafLvrG6SpMs='} 2023-03-24T22:11:16.060 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-03-24_22:00:37-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219596', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-24 22:05:59.290335', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMDwhBowsBXOFxn/w06cj4LLga3mXdGLyNAtME6AQlxc92sSuKpqEFuftyXyhctHuPs9QO+N8Ul6IaQBaIPGK4U='} 2023-03-24T22:11:16.060 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-03-24T22:11:16.066 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-03-24T22:11:16.066 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-03-24T22:11:16.067 INFO:teuthology.run_tasks:Running task console_log... 2023-03-24T22:11:16.153 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1bebdca5e0>, signals=[15]) 2023-03-24T22:11:16.155 INFO:teuthology.run_tasks:Running task internal.connect... 2023-03-24T22:11:16.163 INFO:teuthology.task.internal:Opening connections... 2023-03-24T22:11:16.163 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi018.front.sepia.ceph.com 2023-03-24T22:11:16.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T22:11:16.238 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2023-03-24T22:11:16.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T22:11:16.307 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-03-24T22:11:16.314 DEBUG:teuthology.orchestra.run.smithi018:> uname -m 2023-03-24T22:11:16.337 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2023-03-24T22:11:16.338 DEBUG:teuthology.orchestra.run.smithi018:> cat /etc/os-release 2023-03-24T22:11:16.399 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream" 2023-03-24T22:11:16.400 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="8" 2023-03-24T22:11:16.400 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2023-03-24T22:11:16.400 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2023-03-24T22:11:16.401 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="8" 2023-03-24T22:11:16.401 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el8" 2023-03-24T22:11:16.401 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 8" 2023-03-24T22:11:16.401 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2023-03-24T22:11:16.401 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-03-24T22:11:16.402 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/" 2023-03-24T22:11:16.402 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-24T22:11:16.402 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-24T22:11:16.402 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-03-24T22:11:16.403 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2023-03-24T22:11:16.427 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2023-03-24T22:11:16.448 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2023-03-24T22:11:16.449 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2023-03-24T22:11:16.509 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2023-03-24T22:11:16.509 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="8" 2023-03-24T22:11:16.509 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2023-03-24T22:11:16.509 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2023-03-24T22:11:16.509 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="8" 2023-03-24T22:11:16.510 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el8" 2023-03-24T22:11:16.510 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 8" 2023-03-24T22:11:16.510 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2023-03-24T22:11:16.510 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-03-24T22:11:16.510 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2023-03-24T22:11:16.510 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-24T22:11:16.511 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-24T22:11:16.511 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-03-24T22:11:16.511 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2023-03-24T22:11:16.535 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-03-24T22:11:16.545 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-03-24T22:11:16.551 INFO:teuthology.task.internal:Checking for old test directory... 2023-03-24T22:11:16.552 DEBUG:teuthology.orchestra.run.smithi018:> test '!' -e /home/ubuntu/cephtest 2023-03-24T22:11:16.554 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2023-03-24T22:11:16.573 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-03-24T22:11:16.580 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-03-24T22:11:16.581 DEBUG:teuthology.orchestra.run.smithi018:> test -z $(ls -A /var/lib/ceph) 2023-03-24T22:11:16.615 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2023-03-24T22:11:16.637 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-03-24T22:11:16.702 INFO:teuthology.run_tasks:Running task kernel... 2023-03-24T22:11:16.721 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-03-24T22:11:16.722 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-03-24T22:11:16.722 DEBUG:teuthology.orchestra.run.smithi018:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-24T22:11:16.723 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-24T22:11:16.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-24T22:11:16.743 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2023-03-24T22:11:16.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-24T22:11:16.745 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2023-03-24T22:11:16.803 INFO:teuthology.orchestra.run.smithi100.stdout:4.18.0-408.el8.x86_64 2023-03-24T22:11:16.803 INFO:teuthology.task.kernel:Running kernel on smithi100: 4.18.0-408.el8.x86_64 2023-03-24T22:11:16.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2023-03-24T22:11:16.807 INFO:teuthology.orchestra.run.smithi018.stdout:4.18.0-408.el8.x86_64 2023-03-24T22:11:16.809 INFO:teuthology.task.kernel:Running kernel on smithi018: 4.18.0-408.el8.x86_64 2023-03-24T22:11:16.809 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum install -y kernel 2023-03-24T22:11:20.682 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - AppStream 26 MB/s | 29 MB 00:01 2023-03-24T22:11:20.869 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-8 - AppStream 25 MB/s | 29 MB 00:01 2023-03-24T22:11:26.878 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-8 - BaseOS 34 MB/s | 28 MB 00:00 2023-03-24T22:11:27.100 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - BaseOS 34 MB/s | 28 MB 00:00 2023-03-24T22:11:31.256 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-8 - Extras 119 kB/s | 18 kB 00:00 2023-03-24T22:11:31.754 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-8 - PowerTools 16 MB/s | 5.8 MB 00:00 2023-03-24T22:11:31.948 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - Extras 87 kB/s | 18 kB 00:00 2023-03-24T22:11:32.375 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - PowerTools 20 MB/s | 5.8 MB 00:00 2023-03-24T22:11:33.053 INFO:teuthology.orchestra.run.smithi100.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-03-24T22:11:33.683 INFO:teuthology.orchestra.run.smithi018.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-03-24T22:11:34.579 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 14 MB 00:01 2023-03-24T22:11:35.417 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 14 MB 00:01 2023-03-24T22:11:37.946 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 383 kB/s | 24 kB 00:00 2023-03-24T22:11:39.125 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 455 kB/s | 24 kB 00:00 2023-03-24T22:11:41.185 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-03-24T22:11:41.186 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-03-24T22:11:41.353 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2023-03-24T22:11:41.354 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2023-03-24T22:11:41.354 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2023-03-24T22:11:41.354 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2023-03-24T22:11:41.354 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2023-03-24T22:11:41.354 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-03-24T22:11:41.354 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-03-24T22:11:41.355 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-03-24T22:11:41.355 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:11:41.355 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2023-03-24T22:11:41.355 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2023-03-24T22:11:41.355 INFO:teuthology.orchestra.run.smithi100.stdout:Install 3 Packages 2023-03-24T22:11:41.355 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:11:41.356 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 85 M 2023-03-24T22:11:41.356 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 94 M 2023-03-24T22:11:41.356 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2023-03-24T22:11:41.819 INFO:teuthology.orchestra.run.smithi100.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 20 MB/s | 9.2 MB 00:00 2023-03-24T22:11:42.439 INFO:teuthology.orchestra.run.smithi100.stdout:(2/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 31 MB/s | 34 MB 00:01 2023-03-24T22:11:42.806 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-03-24T22:11:42.807 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-03-24T22:11:43.011 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2023-03-24T22:11:43.012 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2023-03-24T22:11:43.012 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2023-03-24T22:11:43.012 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2023-03-24T22:11:43.013 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2023-03-24T22:11:43.013 INFO:teuthology.orchestra.run.smithi018.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-03-24T22:11:43.013 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-03-24T22:11:43.013 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-03-24T22:11:43.013 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:11:43.013 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2023-03-24T22:11:43.013 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2023-03-24T22:11:43.014 INFO:teuthology.orchestra.run.smithi018.stdout:Install 3 Packages 2023-03-24T22:11:43.014 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:11:43.014 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 85 M 2023-03-24T22:11:43.014 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 94 M 2023-03-24T22:11:43.015 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2023-03-24T22:11:43.097 INFO:teuthology.orchestra.run.smithi100.stdout:(3/3): kernel-core-4.18.0-448.el8.x86_64.rpm 24 MB/s | 41 MB 00:01 2023-03-24T22:11:43.100 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2023-03-24T22:11:43.100 INFO:teuthology.orchestra.run.smithi100.stdout:Total 48 MB/s | 85 MB 00:01 2023-03-24T22:11:43.451 INFO:teuthology.orchestra.run.smithi018.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 21 MB/s | 9.2 MB 00:00 2023-03-24T22:11:43.735 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2023-03-24T22:11:44.124 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2023-03-24T22:11:44.125 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2023-03-24T22:11:44.259 INFO:teuthology.orchestra.run.smithi018.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 33 MB/s | 41 MB 00:01 2023-03-24T22:11:44.600 INFO:teuthology.orchestra.run.smithi018.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 21 MB/s | 34 MB 00:01 2023-03-24T22:11:44.603 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2023-03-24T22:11:44.604 INFO:teuthology.orchestra.run.smithi018.stdout:Total 53 MB/s | 85 MB 00:01 2023-03-24T22:11:45.263 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2023-03-24T22:11:45.685 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2023-03-24T22:11:45.686 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2023-03-24T22:11:46.152 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2023-03-24T22:11:46.177 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2023-03-24T22:11:47.694 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2023-03-24T22:11:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2023-03-24T22:11:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2023-03-24T22:11:49.545 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-24T22:11:50.587 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2023-03-24T22:11:51.181 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-24T22:11:53.425 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-24T22:11:53.870 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-24T22:11:55.152 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-24T22:11:55.614 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-24T22:11:57.181 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-24T22:11:57.436 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-03-24T22:11:59.163 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-24T22:11:59.418 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-03-24T22:12:19.153 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-03-24T22:12:19.180 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-24T22:12:19.604 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-03-24T22:12:19.604 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-03-24T22:12:19.605 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-03-24T22:12:20.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-24T22:12:20.217 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:12:20.217 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2023-03-24T22:12:20.218 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-03-24T22:12:20.218 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-03-24T22:12:20.218 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:12:20.218 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2023-03-24T22:12:20.324 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel --last | head -n 1 2023-03-24T22:12:20.517 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-4.18.0-448.el8.x86_64 Fri 24 Mar 2023 10:11:57 PM UTC 2023-03-24T22:12:20.519 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-24T22:12:20.519 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-03-24T22:12:20.520 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-03-24T22:12:20.520 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-03-24T22:12:20.521 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-03-24T22:12:20.521 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2023-03-24T22:12:20.521 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel --last | head -n 1 2023-03-24T22:12:20.709 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-4.18.0-448.el8.x86_64 Fri 24 Mar 2023 10:11:57 PM UTC 2023-03-24T22:12:20.711 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-24T22:12:20.711 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2023-03-24T22:12:20.743 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2023-03-24T22:12:20.744 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2023-03-24T22:12:20.744 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.02 2023-03-24T22:12:20.744 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 129.el8 2023-03-24T22:12:20.744 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2023-03-24T22:12:20.745 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-03-24T22:12:20.745 INFO:teuthology.orchestra.run.smithi100.stdout:Group : System Environment/Base 2023-03-24T22:12:20.745 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 9265586 2023-03-24T22:12:20.745 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2023-03-24T22:12:20.745 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-03-24T22:12:20.745 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-03-24T22:12:20.745 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-03-24T22:12:20.746 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : kbuilder.bsys.centos.org 2023-03-24T22:12:20.746 INFO:teuthology.orchestra.run.smithi100.stdout:Relocations : (not relocatable) 2023-03-24T22:12:20.746 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : CentOS BuildSystem 2023-03-24T22:12:20.746 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2023-03-24T22:12:20.746 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-24T22:12:20.746 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2023-03-24T22:12:20.746 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2023-03-24T22:12:20.747 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:12:20.747 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-24T22:12:20.747 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-24T22:12:20.747 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-24T22:12:20.747 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2023-03-24T22:12:20.747 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:12:20.747 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2023-03-24T22:12:20.748 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-24T22:12:20.748 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 4.18.0-448.el8.x86_64 2023-03-24T22:12:20.749 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-24T22:12:22.553 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2023-03-24T22:12:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-03-24T22:12:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-24T22:12:23.207 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-03-24T22:12:23.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-03-24T22:12:23.208 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-03-24T22:12:24.354 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-24T22:12:24.355 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:12:24.355 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2023-03-24T22:12:24.355 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-03-24T22:12:24.355 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-03-24T22:12:24.355 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:12:24.355 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2023-03-24T22:12:24.466 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q kernel --last | head -n 1 2023-03-24T22:12:24.667 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-4.18.0-448.el8.x86_64 Fri 24 Mar 2023 10:11:59 PM UTC 2023-03-24T22:12:24.668 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-24T22:12:24.668 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-03-24T22:12:24.669 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-03-24T22:12:24.669 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-03-24T22:12:24.669 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-03-24T22:12:24.669 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi018.front.sepia.ceph.com, path=None, version=distro) 2023-03-24T22:12:24.669 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q kernel --last | head -n 1 2023-03-24T22:12:24.857 INFO:teuthology.orchestra.run.smithi100.stderr:done 2023-03-24T22:12:24.860 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2023-03-24T22:12:24.866 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-4.18.0-448.el8.x86_64 Fri 24 Mar 2023 10:11:59 PM UTC 2023-03-24T22:12:24.867 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-24T22:12:24.868 DEBUG:teuthology.orchestra.run.smithi018:> sudo rpm -qi grub2-tools 2023-03-24T22:12:24.889 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.abVHyfPE0G 2023-03-24T22:12:24.889 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.abVHyfPE0G 2023-03-24T22:12:24.901 INFO:teuthology.orchestra.run.smithi018.stdout:Name : grub2-tools 2023-03-24T22:12:24.902 INFO:teuthology.orchestra.run.smithi018.stdout:Epoch : 1 2023-03-24T22:12:24.902 INFO:teuthology.orchestra.run.smithi018.stdout:Version : 2.02 2023-03-24T22:12:24.902 INFO:teuthology.orchestra.run.smithi018.stdout:Release : 129.el8 2023-03-24T22:12:24.902 INFO:teuthology.orchestra.run.smithi018.stdout:Architecture: x86_64 2023-03-24T22:12:24.902 INFO:teuthology.orchestra.run.smithi018.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-03-24T22:12:24.902 INFO:teuthology.orchestra.run.smithi018.stdout:Group : System Environment/Base 2023-03-24T22:12:24.902 INFO:teuthology.orchestra.run.smithi018.stdout:Size : 9265586 2023-03-24T22:12:24.903 INFO:teuthology.orchestra.run.smithi018.stdout:License : GPLv3+ 2023-03-24T22:12:24.903 INFO:teuthology.orchestra.run.smithi018.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-03-24T22:12:24.903 INFO:teuthology.orchestra.run.smithi018.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-03-24T22:12:24.903 INFO:teuthology.orchestra.run.smithi018.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-03-24T22:12:24.903 INFO:teuthology.orchestra.run.smithi018.stdout:Build Host : kbuilder.bsys.centos.org 2023-03-24T22:12:24.903 INFO:teuthology.orchestra.run.smithi018.stdout:Relocations : (not relocatable) 2023-03-24T22:12:24.903 INFO:teuthology.orchestra.run.smithi018.stdout:Packager : CentOS BuildSystem 2023-03-24T22:12:24.903 INFO:teuthology.orchestra.run.smithi018.stdout:Vendor : CentOS 2023-03-24T22:12:24.904 INFO:teuthology.orchestra.run.smithi018.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-24T22:12:24.904 INFO:teuthology.orchestra.run.smithi018.stdout:Summary : Support tools for GRUB. 2023-03-24T22:12:24.904 INFO:teuthology.orchestra.run.smithi018.stdout:Description : 2023-03-24T22:12:24.904 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:12:24.904 INFO:teuthology.orchestra.run.smithi018.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-24T22:12:24.904 INFO:teuthology.orchestra.run.smithi018.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-24T22:12:24.904 INFO:teuthology.orchestra.run.smithi018.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-24T22:12:24.905 INFO:teuthology.orchestra.run.smithi018.stdout:hardware devices. 2023-03-24T22:12:24.905 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:12:24.905 INFO:teuthology.orchestra.run.smithi018.stdout:This subpackage provides tools for support of all platforms. 2023-03-24T22:12:24.906 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-24T22:12:24.906 INFO:teuthology.task.kernel:Updating grub on smithi018 to boot 4.18.0-448.el8.x86_64 2023-03-24T22:12:24.906 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-24T22:12:24.983 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.abVHyfPE0G 2023-03-24T22:12:25.090 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.abVHyfPE0G is 6KB 2023-03-24T22:12:25.137 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.abVHyfPE0G 2023-03-24T22:12:25.154 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2023-03-24T22:12:25.226 INFO:teuthology.orchestra.run.smithi100.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-03-24T22:12:25.227 INFO:teuthology.orchestra.run.smithi100.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-03-24T22:12:25.227 INFO:teuthology.orchestra.run.smithi100.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-03-24T22:12:25.227 INFO:teuthology.orchestra.run.smithi100.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-03-24T22:12:25.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-03-24T22:12:25.359 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2023-03-24T22:12:26.189 INFO:teuthology.orchestra.run.smithi018.stderr:Generating grub configuration file ... 2023-03-24T22:12:28.250 INFO:teuthology.orchestra.run.smithi018.stderr:done 2023-03-24T22:12:28.252 DEBUG:teuthology.orchestra.run.smithi018:> mktemp 2023-03-24T22:12:28.290 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.2ustgB5Q9a 2023-03-24T22:12:28.291 DEBUG:teuthology.orchestra.run.smithi018:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2ustgB5Q9a 2023-03-24T22:12:28.368 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.2ustgB5Q9a 2023-03-24T22:12:28.519 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.2ustgB5Q9a is 6KB 2023-03-24T22:12:28.567 DEBUG:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.2ustgB5Q9a 2023-03-24T22:12:28.584 DEBUG:teuthology.orchestra.run.smithi018:> sudo /bin/ls /boot/loader/entries || true 2023-03-24T22:12:28.661 INFO:teuthology.orchestra.run.smithi018.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-03-24T22:12:28.661 INFO:teuthology.orchestra.run.smithi018.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-03-24T22:12:28.662 INFO:teuthology.orchestra.run.smithi018.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-03-24T22:12:28.662 INFO:teuthology.orchestra.run.smithi018.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-03-24T22:12:28.663 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-03-24T22:12:28.847 DEBUG:teuthology.orchestra.run.smithi018:> sudo shutdown -r now 2023-03-24T22:12:55.385 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-03-24T22:12:55.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-03-24T22:12:55.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T22:12:58.852 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-03-24T22:12:58.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2023-03-24T22:12:58.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T22:13:13.861 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-03-24T22:13:17.193 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.18 2023-03-24T22:13:43.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-03-24T22:13:43.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T22:13:46.949 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-03-24T22:13:47.194 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2023-03-24T22:13:47.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T22:13:50.245 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.18 2023-03-24T22:14:16.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-03-24T22:14:16.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T22:14:17.321 DEBUG:teuthology.orchestra.run.smithi100:> true 2023-03-24T22:14:17.588 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-03-24T22:14:17.588 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-448.el8.x86_64"... 2023-03-24T22:14:17.589 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2023-03-24T22:14:17.605 INFO:teuthology.orchestra.run.smithi100.stdout:4.18.0-448.el8.x86_64 2023-03-24T22:14:17.606 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-03-24T22:14:17.606 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-24T22:14:17.606 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-03-24T22:14:18.608 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-03-24T22:14:18.608 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-03-24T22:14:18.697 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2023-03-24T22:14:18.713 DEBUG:teuthology.parallel:result is None 2023-03-24T22:14:20.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2023-03-24T22:14:20.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T22:14:20.575 DEBUG:teuthology.orchestra.run.smithi018:> true 2023-03-24T22:14:20.863 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2023-03-24T22:14:20.864 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-448.el8.x86_64"... 2023-03-24T22:14:20.864 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2023-03-24T22:14:20.884 INFO:teuthology.orchestra.run.smithi018.stdout:4.18.0-448.el8.x86_64 2023-03-24T22:14:20.884 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-03-24T22:14:20.884 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-24T22:14:20.884 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-03-24T22:14:21.885 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-03-24T22:14:21.886 DEBUG:teuthology.orchestra.run.smithi018:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-03-24T22:14:21.980 INFO:teuthology.orchestra.run.smithi018.stdout:ttyS1 2023-03-24T22:14:21.991 DEBUG:teuthology.parallel:result is None 2023-03-24T22:14:21.992 INFO:teuthology.run_tasks:Running task internal.base... 2023-03-24T22:14:22.001 INFO:teuthology.task.internal:Creating test directory... 2023-03-24T22:14:22.002 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-24T22:14:22.004 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-24T22:14:22.024 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-03-24T22:14:22.035 INFO:teuthology.run_tasks:Running task internal.archive... 2023-03-24T22:14:22.050 INFO:teuthology.task.internal:Creating archive directory... 2023-03-24T22:14:22.050 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-24T22:14:22.065 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-24T22:14:22.096 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-03-24T22:14:22.107 INFO:teuthology.task.internal:Enabling coredump saving... 2023-03-24T22:14:22.108 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-24T22:14:22.131 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-24T22:14:22.164 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-24T22:14:22.172 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-24T22:14:22.175 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-24T22:14:22.183 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-24T22:14:22.184 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-03-24T22:14:22.195 INFO:teuthology.task.internal:Configuring sudo... 2023-03-24T22:14:22.195 DEBUG:teuthology.orchestra.run.smithi018:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-24T22:14:22.219 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-24T22:14:22.253 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-03-24T22:14:22.266 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-03-24T22:14:22.267 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-24T22:14:22.291 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-24T22:14:22.311 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-24T22:14:22.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-24T22:14:22.446 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-24T22:14:22.535 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-24T22:14:22.577 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:14:22.577 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-24T22:14:22.646 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-24T22:14:22.695 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-24T22:14:22.747 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-24T22:14:22.834 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-24T22:14:22.875 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:14:22.875 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-24T22:14:22.942 DEBUG:teuthology.orchestra.run.smithi018:> sudo service rsyslog restart 2023-03-24T22:14:22.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2023-03-24T22:14:22.997 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-24T22:14:23.045 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-24T22:14:23.330 INFO:teuthology.run_tasks:Running task internal.timer... 2023-03-24T22:14:23.339 INFO:teuthology.task.internal:Starting timer... 2023-03-24T22:14:23.340 INFO:teuthology.run_tasks:Running task pcp... 2023-03-24T22:14:23.357 INFO:teuthology.run_tasks:Running task selinux... 2023-03-24T22:14:23.371 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-03-24T22:14:23.372 DEBUG:teuthology.orchestra.run.smithi018:> sudo service auditd rotate 2023-03-24T22:14:23.451 INFO:teuthology.orchestra.run.smithi018.stdout:Rotating logs: 2023-03-24T22:14:23.454 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2023-03-24T22:14:23.526 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2023-03-24T22:14:23.528 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-03-24T22:14:23.528 DEBUG:teuthology.orchestra.run.smithi018:> /usr/sbin/getenforce 2023-03-24T22:14:23.557 INFO:teuthology.orchestra.run.smithi018.stdout:Permissive 2023-03-24T22:14:23.558 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2023-03-24T22:14:23.581 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2023-03-24T22:14:23.582 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi018.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive'} 2023-03-24T22:14:23.582 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-24T22:14:23.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-24T22:14:23.628 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-24T22:14:23.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-24T22:14:23.656 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-03-24T22:14:23.657 DEBUG:teuthology.orchestra.run.smithi018:> sudo /usr/sbin/setenforce permissive 2023-03-24T22:14:23.720 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2023-03-24T22:14:23.752 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-03-24T22:14:23.763 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-03-24T22:14:23.771 INFO:teuthology.repo_utils:Fetching main from origin 2023-03-24T22:14:23.907 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-03-24T22:14:23.926 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-03-24T22:14:23.928 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi018.front.sepia.ceph.com,smithi100.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-03-24T22:21:45.382 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi018.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com')] 2023-03-24T22:21:45.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2023-03-24T22:21:45.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T22:21:45.456 DEBUG:teuthology.orchestra.run.smithi018:> true 2023-03-24T22:21:45.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-03-24T22:21:45.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-24T22:21:45.592 DEBUG:teuthology.orchestra.run.smithi100:> true 2023-03-24T22:21:45.663 INFO:teuthology.run_tasks:Running task clock... 2023-03-24T22:21:45.678 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-03-24T22:21:45.678 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-24T22:21:45.679 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-24T22:21:45.681 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-24T22:21:45.682 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-24T22:21:45.714 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-24T22:21:45.729 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-24T22:21:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-24T22:21:45.748 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-24T22:21:45.811 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2023-03-24T22:21:45.812 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: ntpd: command not found 2023-03-24T22:21:45.822 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2023-03-24T22:21:45.823 INFO:teuthology.orchestra.run.smithi018.stdout:506 Cannot talk to daemon 2023-03-24T22:21:45.836 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-24T22:21:45.838 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-24T22:21:45.849 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-24T22:21:45.852 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-24T22:21:45.914 INFO:teuthology.orchestra.run.smithi100.stderr:bash: ntpq: command not found 2023-03-24T22:21:45.918 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-24T22:21:45.918 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2023-03-24T22:21:45.918 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T22:21:45.918 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T22:21:45.918 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T22:21:45.918 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T22:21:45.919 INFO:teuthology.orchestra.run.smithi018.stderr:bash: ntpq: command not found 2023-03-24T22:21:45.924 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-24T22:21:45.924 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2023-03-24T22:21:45.924 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T22:21:45.924 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T22:21:45.924 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T22:21:45.924 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-24T22:21:45.925 INFO:teuthology.run_tasks:Running task install... 2023-03-24T22:21:45.935 DEBUG:teuthology.task.install:project ceph 2023-03-24T22:21:45.935 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb'}} 2023-03-24T22:21:45.935 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb'} 2023-03-24T22:21:45.936 INFO:teuthology.task.install:Using flavor: default 2023-03-24T22:21:45.942 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-03-24T22:21:45.943 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-03-24T22:21:45.944 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb', 'tag': None, 'wait_for_package': False} 2023-03-24T22:21:45.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c19e8f2a497e95f75abc43ec14bc6095e30dd2cb 2023-03-24T22:21:45.950 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb', 'tag': None, 'wait_for_package': False} 2023-03-24T22:21:45.950 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c19e8f2a497e95f75abc43ec14bc6095e30dd2cb 2023-03-24T22:21:46.090 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/ 2023-03-24T22:21:46.091 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3124.gc19e8f2a 2023-03-24T22:21:46.099 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/ 2023-03-24T22:21:46.099 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3124.gc19e8f2a 2023-03-24T22:21:46.195 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-24T22:21:46.195 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:21:46.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-24T22:21:46.200 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230323.162417/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-24T22:21:46.200 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:21:46.200 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-24T22:21:46.225 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-03-24T22:21:46.225 DEBUG:teuthology.orchestra.run.smithi018:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-24T22:21:46.228 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-03-24T22:21:46.228 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c19e8f2a497e95f75abc43ec14bc6095e30dd2cb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-24T22:21:46.295 DEBUG:teuthology.orchestra.run.smithi018:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-24T22:21:46.296 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-24T22:21:46.373 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-24T22:21:46.374 DEBUG:teuthology.orchestra.run.smithi018:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-24T22:21:46.439 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2023-03-24T22:21:46.441 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2023-03-24T22:21:46.441 INFO:teuthology.orchestra.run.smithi018.stdout:check_obsoletes = 1 2023-03-24T22:21:46.442 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum clean all 2023-03-24T22:21:46.692 INFO:teuthology.orchestra.run.smithi100.stdout:71 files removed 2023-03-24T22:21:46.707 INFO:teuthology.orchestra.run.smithi018.stdout:71 files removed 2023-03-24T22:21:46.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-03-24T22:21:46.737 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-03-24T22:21:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 307 kB/s | 80 kB 00:00 2023-03-24T22:21:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:ceph packages for x86_64 310 kB/s | 80 kB 00:00 2023-03-24T22:21:47.660 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 40 kB/s | 9.4 kB 00:00 2023-03-24T22:21:47.683 INFO:teuthology.orchestra.run.smithi018.stdout:ceph noarch packages 43 kB/s | 9.4 kB 00:00 2023-03-24T22:21:47.911 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 6.7 kB/s | 1.5 kB 00:00 2023-03-24T22:21:47.950 INFO:teuthology.orchestra.run.smithi018.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2023-03-24T22:21:48.614 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-8 - AppStream 42 MB/s | 29 MB 00:00 2023-03-24T22:21:48.828 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - AppStream 34 MB/s | 29 MB 00:00 2023-03-24T22:21:54.523 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-8 - BaseOS 36 MB/s | 28 MB 00:00 2023-03-24T22:21:55.070 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - BaseOS 41 MB/s | 28 MB 00:00 2023-03-24T22:21:58.938 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-8 - Extras 109 kB/s | 18 kB 00:00 2023-03-24T22:21:59.349 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-8 - PowerTools 21 MB/s | 5.8 MB 00:00 2023-03-24T22:21:59.702 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - Extras 153 kB/s | 18 kB 00:00 2023-03-24T22:22:00.113 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - PowerTools 22 MB/s | 5.8 MB 00:00 2023-03-24T22:22:00.638 INFO:teuthology.orchestra.run.smithi100.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-03-24T22:22:01.428 INFO:teuthology.orchestra.run.smithi018.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-03-24T22:22:01.774 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 14 MB/s | 14 MB 00:01 2023-03-24T22:22:02.788 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 11 MB/s | 14 MB 00:01 2023-03-24T22:22:05.266 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 393 kB/s | 24 kB 00:00 2023-03-24T22:22:06.554 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 526 kB/s | 24 kB 00:00 2023-03-24T22:22:08.863 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-24T22:22:08.863 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-24T22:22:08.875 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.39.12-1.el8.x86_64 is already installed. 2023-03-24T22:22:08.875 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-initscripts-updown-1:1.39.12-1.el8.noarch is already installed. 2023-03-24T22:22:08.876 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-5.18.0-1.el8.x86_64 is already installed. 2023-03-24T22:22:08.877 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.32.1-38.el8.x86_64 is already installed. 2023-03-24T22:22:08.878 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-03-24T22:22:08.880 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-03-24T22:22:08.883 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-03-24T22:22:08.883 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-03-24T22:22:09.015 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2023-03-24T22:22:09.029 INFO:teuthology.orchestra.run.smithi100.stdout:========================================================================================================= 2023-03-24T22:22:09.029 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2023-03-24T22:22:09.029 INFO:teuthology.orchestra.run.smithi100.stdout:========================================================================================================= 2023-03-24T22:22:09.029 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2023-03-24T22:22:09.029 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.0.4-10.el8 CentOS-AppStream 688 k 2023-03-24T22:22:09.030 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 6.4 k 2023-03-24T22:22:09.030 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 5.4 M 2023-03-24T22:22:09.030 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 846 k 2023-03-24T22:22:09.030 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 165 k 2023-03-24T22:22:09.030 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 1.5 M 2023-03-24T22:22:09.030 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 123 k 2023-03-24T22:22:09.030 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 1.7 M 2023-03-24T22:22:09.031 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 7.4 M 2023-03-24T22:22:09.031 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 47 k 2023-03-24T22:22:09.031 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 13 M 2023-03-24T22:22:09.031 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 41 M 2023-03-24T22:22:09.032 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 263 k 2023-03-24T22:22:09.032 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 83 k 2023-03-24T22:22:09.032 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 258 k 2023-03-24T22:22:09.032 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 18 k 2023-03-24T22:22:09.032 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.188-3.el8 CentOS-BaseOS 61 k 2023-03-24T22:22:09.033 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.1-9.el8 CentOS-AppStream 320 k 2023-03-24T22:22:09.033 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 19 k 2023-03-24T22:22:09.033 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 30 k 2023-03-24T22:22:09.033 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 769 k 2023-03-24T22:22:09.034 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 139 k 2023-03-24T22:22:09.034 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-25.el8 CentOS-AppStream 709 k 2023-03-24T22:22:09.034 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.32.1-41.el8 CentOS-BaseOS 99 k 2023-03-24T22:22:09.035 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 3.2 M 2023-03-24T22:22:09.035 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 1.6 M 2023-03-24T22:22:09.035 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 903 k 2023-03-24T22:22:09.035 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 5.1 M 2023-03-24T22:22:09.035 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 190 k 2023-03-24T22:22:09.035 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 370 k 2023-03-24T22:22:09.035 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 364 k 2023-03-24T22:22:09.036 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 107 k 2023-03-24T22:22:09.036 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 94 k 2023-03-24T22:22:09.036 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.4 M 2023-03-24T22:22:09.036 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 179 k 2023-03-24T22:22:09.036 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 CentOS-BaseOS 72 k 2023-03-24T22:22:09.036 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2023-03-24T22:22:09.036 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-1.el8 CentOS-BaseOS 142 k 2023-03-24T22:22:09.037 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client x86_64 0.188-3.el8 CentOS-BaseOS 75 k 2023-03-24T22:22:09.037 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf x86_64 0.188-3.el8 CentOS-BaseOS 231 k 2023-03-24T22:22:09.037 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs x86_64 0.188-3.el8 CentOS-BaseOS 299 k 2023-03-24T22:22:09.037 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid x86_64 2.32.1-41.el8 CentOS-BaseOS 220 k 2023-03-24T22:22:09.037 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk x86_64 2.32.1-41.el8 CentOS-BaseOS 253 k 2023-03-24T22:22:09.037 INFO:teuthology.orchestra.run.smithi100.stdout: libmount x86_64 2.32.1-41.el8 CentOS-BaseOS 236 k 2023-03-24T22:22:09.037 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.7 M 2023-03-24T22:22:09.038 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.6 M 2023-03-24T22:22:09.038 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols x86_64 2.32.1-41.el8 CentOS-BaseOS 179 k 2023-03-24T22:22:09.038 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid x86_64 2.32.1-41.el8 CentOS-BaseOS 98 k 2023-03-24T22:22:09.038 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 737 k 2023-03-24T22:22:09.038 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 1.5 M 2023-03-24T22:22:09.038 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux x86_64 2.32.1-41.el8 CentOS-BaseOS 2.6 M 2023-03-24T22:22:09.038 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user x86_64 2.32.1-41.el8 CentOS-BaseOS 102 k 2023-03-24T22:22:09.038 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2023-03-24T22:22:09.039 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.1-7.el8 CentOS-AppStream 713 k 2023-03-24T22:22:09.039 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 22 M 2023-03-24T22:22:09.039 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 23 k 2023-03-24T22:22:09.039 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 2.3 M 2023-03-24T22:22:09.039 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 239 k 2023-03-24T22:22:09.039 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 4.6 M 2023-03-24T22:22:09.039 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 18 M 2023-03-24T22:22:09.040 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 15 k 2023-03-24T22:22:09.040 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 24 k 2023-03-24T22:22:09.040 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-03-24T22:22:09.040 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-24T22:22:09.040 INFO:teuthology.orchestra.run.smithi100.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-03-24T22:22:09.040 INFO:teuthology.orchestra.run.smithi100.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 CentOS-BaseOS 48 k 2023-03-24T22:22:09.040 INFO:teuthology.orchestra.run.smithi100.stdout: krb5-devel x86_64 1.18.2-22.el8 CentOS-BaseOS 561 k 2023-03-24T22:22:09.040 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 137 k 2023-03-24T22:22:09.041 INFO:teuthology.orchestra.run.smithi100.stdout: libcom_err-devel x86_64 1.45.6-5.el8 CentOS-BaseOS 39 k 2023-03-24T22:22:09.041 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-03-24T22:22:09.041 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-24T22:22:09.041 INFO:teuthology.orchestra.run.smithi100.stdout: libpq x86_64 13.5-1.el8 CentOS-AppStream 198 k 2023-03-24T22:22:09.041 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-03-24T22:22:09.041 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-03-24T22:22:09.041 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 474 k 2023-03-24T22:22:09.042 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-03-24T22:22:09.042 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 6.5 M 2023-03-24T22:22:09.042 INFO:teuthology.orchestra.run.smithi100.stdout: libselinux-devel x86_64 2.9-6.el8 CentOS-BaseOS 200 k 2023-03-24T22:22:09.042 INFO:teuthology.orchestra.run.smithi100.stdout: libsepol-devel x86_64 2.9-3.el8 CentOS-BaseOS 87 k 2023-03-24T22:22:09.042 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-24T22:22:09.042 INFO:teuthology.orchestra.run.smithi100.stdout: libverto-devel x86_64 0.3.2-2.el8 CentOS-BaseOS 18 k 2023-03-24T22:22:09.043 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-03-24T22:22:09.043 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-03-24T22:22:09.043 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-03-24T22:22:09.043 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-03-24T22:22:09.043 INFO:teuthology.orchestra.run.smithi100.stdout: pcre2-devel x86_64 10.32-3.el8 CentOS-BaseOS 605 k 2023-03-24T22:22:09.043 INFO:teuthology.orchestra.run.smithi100.stdout: pcre2-utf16 x86_64 10.32-3.el8 CentOS-BaseOS 229 k 2023-03-24T22:22:09.043 INFO:teuthology.orchestra.run.smithi100.stdout: pcre2-utf32 x86_64 10.32-3.el8 CentOS-BaseOS 220 k 2023-03-24T22:22:09.043 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-24T22:22:09.044 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-03-24T22:22:09.044 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-24T22:22:09.044 INFO:teuthology.orchestra.run.smithi100.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-24T22:22:09.044 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-24T22:22:09.044 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 45 k 2023-03-24T22:22:09.044 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 117 k 2023-03-24T22:22:09.044 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-24T22:22:09.045 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-03-24T22:22:09.045 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-24T22:22:09.045 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-24T22:22:09.045 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-03-24T22:22:09.045 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-24T22:22:09.046 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-24T22:22:09.046 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-24T22:22:09.046 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-03-24T22:22:09.046 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-03-24T22:22:09.046 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-03-24T22:22:09.046 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-03-24T22:22:09.047 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-24T22:22:09.047 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-24T22:22:09.047 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.0.6-14.el8 CentOS-AppStream 157 k 2023-03-24T22:22:09.047 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-03-24T22:22:09.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-24T22:22:09.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-24T22:22:09.048 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-03-24T22:22:09.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-03-24T22:22:09.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-03-24T22:22:09.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-24T22:22:09.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-24T22:22:09.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-03-24T22:22:09.049 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-03-24T22:22:09.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-03-24T22:22:09.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-03-24T22:22:09.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-03-24T22:22:09.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-03-24T22:22:09.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-24T22:22:09.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-03-24T22:22:09.050 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-24T22:22:09.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-24T22:22:09.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-03-24T22:22:09.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-24T22:22:09.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-24T22:22:09.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-24T22:22:09.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-24T22:22:09.051 INFO:teuthology.orchestra.run.smithi100.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-24T22:22:09.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-24T22:22:09.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-24T22:22:09.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-24T22:22:09.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-03-24T22:22:09.052 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-24T22:22:09.052 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-03-24T22:22:09.052 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-24T22:22:09.052 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-43.el8 CentOS-AppStream 64 k 2023-03-24T22:22:09.053 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-03-24T22:22:09.053 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-19.el8 CentOS-BaseOS 58 k 2023-03-24T22:22:09.053 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling module streams: 2023-03-24T22:22:09.053 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql 10 2023-03-24T22:22:09.053 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:22:09.053 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2023-03-24T22:22:09.053 INFO:teuthology.orchestra.run.smithi100.stdout:========================================================================================================= 2023-03-24T22:22:09.054 INFO:teuthology.orchestra.run.smithi100.stdout:Install 127 Packages 2023-03-24T22:22:09.054 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 15 Packages 2023-03-24T22:22:09.054 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:22:09.054 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 204 M 2023-03-24T22:22:09.055 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2023-03-24T22:22:09.176 INFO:teuthology.orchestra.run.smithi100.stdout:(1/142): ceph-18.0.0-3124.gc19e8f2a.el8.x86_64. 50 kB/s | 6.4 kB 00:00 2023-03-24T22:22:09.467 INFO:teuthology.orchestra.run.smithi100.stdout:(2/142): ceph-fuse-18.0.0-3124.gc19e8f2a.el8.x8 2.8 MB/s | 846 kB 00:00 2023-03-24T22:22:09.535 INFO:teuthology.orchestra.run.smithi100.stdout:(3/142): ceph-immutable-object-cache-18.0.0-312 2.4 MB/s | 165 kB 00:00 2023-03-24T22:22:09.668 INFO:teuthology.orchestra.run.smithi100.stdout:(4/142): ceph-base-18.0.0-3124.gc19e8f2a.el8.x8 8.6 MB/s | 5.4 MB 00:00 2023-03-24T22:22:09.868 INFO:teuthology.orchestra.run.smithi100.stdout:(5/142): ceph-mgr-18.0.0-3124.gc19e8f2a.el8.x86 7.7 MB/s | 1.5 MB 00:00 2023-03-24T22:22:10.084 INFO:teuthology.orchestra.run.smithi100.stdout:(6/142): ceph-mds-18.0.0-3124.gc19e8f2a.el8.x86 4.1 MB/s | 2.3 MB 00:00 2023-03-24T22:22:10.301 INFO:teuthology.orchestra.run.smithi018.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-24T22:22:10.302 INFO:teuthology.orchestra.run.smithi018.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-24T22:22:10.315 INFO:teuthology.orchestra.run.smithi018.stdout:Package NetworkManager-1:1.39.12-1.el8.x86_64 is already installed. 2023-03-24T22:22:10.315 INFO:teuthology.orchestra.run.smithi018.stdout:Package NetworkManager-initscripts-updown-1:1.39.12-1.el8.noarch is already installed. 2023-03-24T22:22:10.316 INFO:teuthology.orchestra.run.smithi018.stdout:Package iproute-5.18.0-1.el8.x86_64 is already installed. 2023-03-24T22:22:10.317 INFO:teuthology.orchestra.run.smithi018.stdout:Package util-linux-2.32.1-38.el8.x86_64 is already installed. 2023-03-24T22:22:10.318 INFO:teuthology.orchestra.run.smithi018.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-03-24T22:22:10.320 INFO:teuthology.orchestra.run.smithi018.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-03-24T22:22:10.322 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-03-24T22:22:10.323 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-03-24T22:22:10.369 INFO:teuthology.orchestra.run.smithi100.stdout:(7/142): ceph-mon-18.0.0-3124.gc19e8f2a.el8.x86 9.2 MB/s | 4.6 MB 00:00 2023-03-24T22:22:10.473 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2023-03-24T22:22:10.488 INFO:teuthology.orchestra.run.smithi018.stdout:========================================================================================================= 2023-03-24T22:22:10.489 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2023-03-24T22:22:10.489 INFO:teuthology.orchestra.run.smithi018.stdout:========================================================================================================= 2023-03-24T22:22:10.489 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2023-03-24T22:22:10.489 INFO:teuthology.orchestra.run.smithi018.stdout: bison x86_64 3.0.4-10.el8 CentOS-AppStream 688 k 2023-03-24T22:22:10.489 INFO:teuthology.orchestra.run.smithi018.stdout: ceph x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 6.4 k 2023-03-24T22:22:10.489 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 5.4 M 2023-03-24T22:22:10.489 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 846 k 2023-03-24T22:22:10.490 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 165 k 2023-03-24T22:22:10.490 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 1.5 M 2023-03-24T22:22:10.490 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 123 k 2023-03-24T22:22:10.490 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 1.7 M 2023-03-24T22:22:10.490 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 7.4 M 2023-03-24T22:22:10.490 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 47 k 2023-03-24T22:22:10.490 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 13 M 2023-03-24T22:22:10.491 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 41 M 2023-03-24T22:22:10.491 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 263 k 2023-03-24T22:22:10.491 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 83 k 2023-03-24T22:22:10.491 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-mirror x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 258 k 2023-03-24T22:22:10.491 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-top noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 18 k 2023-03-24T22:22:10.491 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel x86_64 0.188-3.el8 CentOS-BaseOS 61 k 2023-03-24T22:22:10.491 INFO:teuthology.orchestra.run.smithi018.stdout: flex x86_64 2.6.1-9.el8 CentOS-AppStream 320 k 2023-03-24T22:22:10.491 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 19 k 2023-03-24T22:22:10.492 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 30 k 2023-03-24T22:22:10.492 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 769 k 2023-03-24T22:22:10.492 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 139 k 2023-03-24T22:22:10.492 INFO:teuthology.orchestra.run.smithi018.stdout: libtool x86_64 2.4.6-25.el8 CentOS-AppStream 709 k 2023-03-24T22:22:10.492 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel x86_64 2.32.1-41.el8 CentOS-BaseOS 99 k 2023-03-24T22:22:10.492 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 3.2 M 2023-03-24T22:22:10.492 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 1.6 M 2023-03-24T22:22:10.493 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 903 k 2023-03-24T22:22:10.493 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 5.1 M 2023-03-24T22:22:10.493 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 190 k 2023-03-24T22:22:10.493 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 370 k 2023-03-24T22:22:10.493 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 364 k 2023-03-24T22:22:10.493 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 107 k 2023-03-24T22:22:10.493 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 94 k 2023-03-24T22:22:10.493 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.4 M 2023-03-24T22:22:10.494 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 179 k 2023-03-24T22:22:10.494 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 CentOS-BaseOS 72 k 2023-03-24T22:22:10.494 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrading: 2023-03-24T22:22:10.494 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-1.el8 CentOS-BaseOS 142 k 2023-03-24T22:22:10.494 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-debuginfod-client x86_64 0.188-3.el8 CentOS-BaseOS 75 k 2023-03-24T22:22:10.494 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf x86_64 0.188-3.el8 CentOS-BaseOS 231 k 2023-03-24T22:22:10.494 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libs x86_64 0.188-3.el8 CentOS-BaseOS 299 k 2023-03-24T22:22:10.495 INFO:teuthology.orchestra.run.smithi018.stdout: libblkid x86_64 2.32.1-41.el8 CentOS-BaseOS 220 k 2023-03-24T22:22:10.495 INFO:teuthology.orchestra.run.smithi018.stdout: libfdisk x86_64 2.32.1-41.el8 CentOS-BaseOS 253 k 2023-03-24T22:22:10.495 INFO:teuthology.orchestra.run.smithi018.stdout: libmount x86_64 2.32.1-41.el8 CentOS-BaseOS 236 k 2023-03-24T22:22:10.495 INFO:teuthology.orchestra.run.smithi018.stdout: librados2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.7 M 2023-03-24T22:22:10.495 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 3.6 M 2023-03-24T22:22:10.495 INFO:teuthology.orchestra.run.smithi018.stdout: libsmartcols x86_64 2.32.1-41.el8 CentOS-BaseOS 179 k 2023-03-24T22:22:10.495 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid x86_64 2.32.1-41.el8 CentOS-BaseOS 98 k 2023-03-24T22:22:10.495 INFO:teuthology.orchestra.run.smithi018.stdout: openssl x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 737 k 2023-03-24T22:22:10.496 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 1.5 M 2023-03-24T22:22:10.496 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux x86_64 2.32.1-41.el8 CentOS-BaseOS 2.6 M 2023-03-24T22:22:10.496 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-user x86_64 2.32.1-41.el8 CentOS-BaseOS 102 k 2023-03-24T22:22:10.496 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2023-03-24T22:22:10.496 INFO:teuthology.orchestra.run.smithi018.stdout: automake noarch 1.16.1-7.el8 CentOS-AppStream 713 k 2023-03-24T22:22:10.496 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 22 M 2023-03-24T22:22:10.496 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 23 k 2023-03-24T22:22:10.497 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 2.3 M 2023-03-24T22:22:10.497 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 239 k 2023-03-24T22:22:10.497 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 4.6 M 2023-03-24T22:22:10.497 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 18 M 2023-03-24T22:22:10.497 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3124.gc19e8f2a.el8 ceph-noarch 15 k 2023-03-24T22:22:10.497 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 24 k 2023-03-24T22:22:10.497 INFO:teuthology.orchestra.run.smithi018.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-03-24T22:22:10.497 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-24T22:22:10.498 INFO:teuthology.orchestra.run.smithi018.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-03-24T22:22:10.498 INFO:teuthology.orchestra.run.smithi018.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 CentOS-BaseOS 48 k 2023-03-24T22:22:10.498 INFO:teuthology.orchestra.run.smithi018.stdout: krb5-devel x86_64 1.18.2-22.el8 CentOS-BaseOS 561 k 2023-03-24T22:22:10.498 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 137 k 2023-03-24T22:22:10.498 INFO:teuthology.orchestra.run.smithi018.stdout: libcom_err-devel x86_64 1.45.6-5.el8 CentOS-BaseOS 39 k 2023-03-24T22:22:10.498 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-03-24T22:22:10.498 INFO:teuthology.orchestra.run.smithi018.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-24T22:22:10.499 INFO:teuthology.orchestra.run.smithi018.stdout: libpq x86_64 13.5-1.el8 CentOS-AppStream 198 k 2023-03-24T22:22:10.499 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-03-24T22:22:10.499 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-03-24T22:22:10.499 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 474 k 2023-03-24T22:22:10.499 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-03-24T22:22:10.499 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2 x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 6.5 M 2023-03-24T22:22:10.499 INFO:teuthology.orchestra.run.smithi018.stdout: libselinux-devel x86_64 2.9-6.el8 CentOS-BaseOS 200 k 2023-03-24T22:22:10.500 INFO:teuthology.orchestra.run.smithi018.stdout: libsepol-devel x86_64 2.9-3.el8 CentOS-BaseOS 87 k 2023-03-24T22:22:10.500 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-24T22:22:10.500 INFO:teuthology.orchestra.run.smithi018.stdout: libverto-devel x86_64 0.3.2-2.el8 CentOS-BaseOS 18 k 2023-03-24T22:22:10.500 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-03-24T22:22:10.500 INFO:teuthology.orchestra.run.smithi018.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-03-24T22:22:10.500 INFO:teuthology.orchestra.run.smithi018.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-03-24T22:22:10.500 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-03-24T22:22:10.500 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-devel x86_64 10.32-3.el8 CentOS-BaseOS 605 k 2023-03-24T22:22:10.501 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-utf16 x86_64 10.32-3.el8 CentOS-BaseOS 229 k 2023-03-24T22:22:10.501 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-utf32 x86_64 10.32-3.el8 CentOS-BaseOS 220 k 2023-03-24T22:22:10.501 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-24T22:22:10.501 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-03-24T22:22:10.501 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-24T22:22:10.501 INFO:teuthology.orchestra.run.smithi018.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-24T22:22:10.501 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-24T22:22:10.502 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 45 k 2023-03-24T22:22:10.502 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common x86_64 2:18.0.0-3124.gc19e8f2a.el8 ceph 117 k 2023-03-24T22:22:10.502 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-24T22:22:10.502 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-03-24T22:22:10.502 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-24T22:22:10.502 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-24T22:22:10.502 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-03-24T22:22:10.502 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-24T22:22:10.503 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-24T22:22:10.503 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-24T22:22:10.503 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-03-24T22:22:10.503 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-03-24T22:22:10.503 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-03-24T22:22:10.503 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-03-24T22:22:10.503 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-24T22:22:10.503 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-24T22:22:10.504 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako noarch 1.0.6-14.el8 CentOS-AppStream 157 k 2023-03-24T22:22:10.504 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-03-24T22:22:10.504 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-24T22:22:10.504 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-24T22:22:10.504 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-03-24T22:22:10.504 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-03-24T22:22:10.504 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-03-24T22:22:10.505 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-24T22:22:10.505 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-24T22:22:10.505 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-03-24T22:22:10.505 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-03-24T22:22:10.505 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-03-24T22:22:10.505 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-03-24T22:22:10.505 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-03-24T22:22:10.505 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-03-24T22:22:10.506 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-24T22:22:10.506 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-03-24T22:22:10.506 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-24T22:22:10.506 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-24T22:22:10.506 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-03-24T22:22:10.506 INFO:teuthology.orchestra.run.smithi018.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-24T22:22:10.506 INFO:teuthology.orchestra.run.smithi018.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-24T22:22:10.507 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-24T22:22:10.507 INFO:teuthology.orchestra.run.smithi018.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-24T22:22:10.507 INFO:teuthology.orchestra.run.smithi018.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-24T22:22:10.507 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-24T22:22:10.507 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-24T22:22:10.507 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-24T22:22:10.507 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-03-24T22:22:10.507 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-24T22:22:10.508 INFO:teuthology.orchestra.run.smithi018.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-03-24T22:22:10.508 INFO:teuthology.orchestra.run.smithi018.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-24T22:22:10.508 INFO:teuthology.orchestra.run.smithi018.stdout: uuid x86_64 1.6.2-43.el8 CentOS-AppStream 64 k 2023-03-24T22:22:10.508 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-03-24T22:22:10.508 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel x86_64 1.2.11-19.el8 CentOS-BaseOS 58 k 2023-03-24T22:22:10.508 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling module streams: 2023-03-24T22:22:10.508 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql 10 2023-03-24T22:22:10.509 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:22:10.509 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2023-03-24T22:22:10.509 INFO:teuthology.orchestra.run.smithi018.stdout:========================================================================================================= 2023-03-24T22:22:10.509 INFO:teuthology.orchestra.run.smithi018.stdout:Install 127 Packages 2023-03-24T22:22:10.509 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrade 15 Packages 2023-03-24T22:22:10.509 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:22:10.510 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 204 M 2023-03-24T22:22:10.510 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2023-03-24T22:22:10.780 INFO:teuthology.orchestra.run.smithi018.stdout:(1/142): ceph-18.0.0-3124.gc19e8f2a.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2023-03-24T22:22:10.969 INFO:teuthology.orchestra.run.smithi100.stdout:(8/142): ceph-common-18.0.0-3124.gc19e8f2a.el8. 11 MB/s | 22 MB 00:01 2023-03-24T22:22:11.036 INFO:teuthology.orchestra.run.smithi100.stdout:(9/142): ceph-selinux-18.0.0-3124.gc19e8f2a.el8 366 kB/s | 24 kB 00:00 2023-03-24T22:22:11.239 INFO:teuthology.orchestra.run.smithi018.stdout:(2/142): ceph-fuse-18.0.0-3124.gc19e8f2a.el8.x8 1.8 MB/s | 846 kB 00:00 2023-03-24T22:22:11.414 INFO:teuthology.orchestra.run.smithi018.stdout:(3/142): ceph-immutable-object-cache-18.0.0-312 945 kB/s | 165 kB 00:00 2023-03-24T22:22:12.061 INFO:teuthology.orchestra.run.smithi100.stdout:(10/142): ceph-radosgw-18.0.0-3124.gc19e8f2a.el 7.7 MB/s | 13 MB 00:01 2023-03-24T22:22:12.170 INFO:teuthology.orchestra.run.smithi100.stdout:(11/142): cephfs-mirror-18.0.0-3124.gc19e8f2a.e 2.3 MB/s | 258 kB 00:00 2023-03-24T22:22:12.253 INFO:teuthology.orchestra.run.smithi100.stdout:(12/142): libcephfs-devel-18.0.0-3124.gc19e8f2a 365 kB/s | 30 kB 00:00 2023-03-24T22:22:12.473 INFO:teuthology.orchestra.run.smithi018.stdout:(4/142): ceph-mds-18.0.0-3124.gc19e8f2a.el8.x86 2.2 MB/s | 2.3 MB 00:01 2023-03-24T22:22:12.489 INFO:teuthology.orchestra.run.smithi100.stdout:(13/142): libcephfs2-18.0.0-3124.gc19e8f2a.el8. 3.2 MB/s | 769 kB 00:00 2023-03-24T22:22:12.579 INFO:teuthology.orchestra.run.smithi100.stdout:(14/142): libcephsqlite-18.0.0-3124.gc19e8f2a.e 1.5 MB/s | 137 kB 00:00 2023-03-24T22:22:12.671 INFO:teuthology.orchestra.run.smithi100.stdout:(15/142): librados-devel-18.0.0-3124.gc19e8f2a. 1.5 MB/s | 139 kB 00:00 2023-03-24T22:22:12.846 INFO:teuthology.orchestra.run.smithi100.stdout:(16/142): libradosstriper1-18.0.0-3124.gc19e8f2 2.6 MB/s | 474 kB 00:00 2023-03-24T22:22:13.206 INFO:teuthology.orchestra.run.smithi018.stdout:(5/142): ceph-mgr-18.0.0-3124.gc19e8f2a.el8.x86 2.1 MB/s | 1.5 MB 00:00 2023-03-24T22:22:13.382 INFO:teuthology.orchestra.run.smithi018.stdout:(6/142): ceph-base-18.0.0-3124.gc19e8f2a.el8.x8 1.9 MB/s | 5.4 MB 00:02 2023-03-24T22:22:13.815 INFO:teuthology.orchestra.run.smithi018.stdout:(7/142): ceph-common-18.0.0-3124.gc19e8f2a.el8. 6.5 MB/s | 22 MB 00:03 2023-03-24T22:22:14.163 INFO:teuthology.orchestra.run.smithi100.stdout:(17/142): librgw2-18.0.0-3124.gc19e8f2a.el8.x86 4.9 MB/s | 6.5 MB 00:01 2023-03-24T22:22:14.247 INFO:teuthology.orchestra.run.smithi100.stdout:(18/142): python3-ceph-argparse-18.0.0-3124.gc1 535 kB/s | 45 kB 00:00 2023-03-24T22:22:14.339 INFO:teuthology.orchestra.run.smithi100.stdout:(19/142): python3-ceph-common-18.0.0-3124.gc19e 1.2 MB/s | 117 kB 00:00 2023-03-24T22:22:14.439 INFO:teuthology.orchestra.run.smithi100.stdout:(20/142): python3-cephfs-18.0.0-3124.gc19e8f2a. 1.9 MB/s | 190 kB 00:00 2023-03-24T22:22:14.706 INFO:teuthology.orchestra.run.smithi100.stdout:(21/142): ceph-osd-18.0.0-3124.gc19e8f2a.el8.x8 3.8 MB/s | 18 MB 00:04 2023-03-24T22:22:14.731 INFO:teuthology.orchestra.run.smithi100.stdout:(22/142): python3-rados-18.0.0-3124.gc19e8f2a.e 1.2 MB/s | 370 kB 00:00 2023-03-24T22:22:14.823 INFO:teuthology.orchestra.run.smithi100.stdout:(23/142): python3-rgw-18.0.0-3124.gc19e8f2a.el8 1.1 MB/s | 107 kB 00:00 2023-03-24T22:22:14.949 INFO:teuthology.orchestra.run.smithi100.stdout:(24/142): rbd-fuse-18.0.0-3124.gc19e8f2a.el8.x8 755 kB/s | 94 kB 00:00 2023-03-24T22:22:14.974 INFO:teuthology.orchestra.run.smithi100.stdout:(25/142): python3-rbd-18.0.0-3124.gc19e8f2a.el8 1.3 MB/s | 364 kB 00:00 2023-03-24T22:22:15.066 INFO:teuthology.orchestra.run.smithi100.stdout:(26/142): rbd-nbd-18.0.0-3124.gc19e8f2a.el8.x86 1.9 MB/s | 179 kB 00:00 2023-03-24T22:22:15.099 INFO:teuthology.orchestra.run.smithi018.stdout:(8/142): ceph-mon-18.0.0-3124.gc19e8f2a.el8.x86 2.4 MB/s | 4.6 MB 00:01 2023-03-24T22:22:15.141 INFO:teuthology.orchestra.run.smithi100.stdout:(27/142): ceph-grafana-dashboards-18.0.0-3124.g 308 kB/s | 23 kB 00:00 2023-03-24T22:22:15.174 INFO:teuthology.orchestra.run.smithi018.stdout:(9/142): ceph-selinux-18.0.0-3124.gc19e8f2a.el8 324 kB/s | 24 kB 00:00 2023-03-24T22:22:15.358 INFO:teuthology.orchestra.run.smithi018.stdout:(10/142): ceph-radosgw-18.0.0-3124.gc19e8f2a.el 8.5 MB/s | 13 MB 00:01 2023-03-24T22:22:15.436 INFO:teuthology.orchestra.run.smithi018.stdout:(11/142): cephfs-mirror-18.0.0-3124.gc19e8f2a.e 3.2 MB/s | 258 kB 00:00 2023-03-24T22:22:15.491 INFO:teuthology.orchestra.run.smithi018.stdout:(12/142): libcephfs-devel-18.0.0-3124.gc19e8f2a 553 kB/s | 30 kB 00:00 2023-03-24T22:22:15.525 INFO:teuthology.orchestra.run.smithi100.stdout:(28/142): ceph-test-18.0.0-3124.gc19e8f2a.el8.x 9.0 MB/s | 41 MB 00:04 2023-03-24T22:22:15.550 INFO:teuthology.orchestra.run.smithi100.stdout:(29/142): ceph-mgr-cephadm-18.0.0-3124.gc19e8f2 302 kB/s | 123 kB 00:00 2023-03-24T22:22:15.600 INFO:teuthology.orchestra.run.smithi018.stdout:(13/142): libcephfs2-18.0.0-3124.gc19e8f2a.el8. 6.9 MB/s | 769 kB 00:00 2023-03-24T22:22:15.684 INFO:teuthology.orchestra.run.smithi018.stdout:(14/142): libcephsqlite-18.0.0-3124.gc19e8f2a.e 1.6 MB/s | 137 kB 00:00 2023-03-24T22:22:15.742 INFO:teuthology.orchestra.run.smithi100.stdout:(30/142): ceph-mgr-dashboard-18.0.0-3124.gc19e8 7.8 MB/s | 1.7 MB 00:00 2023-03-24T22:22:15.743 INFO:teuthology.orchestra.run.smithi018.stdout:(15/142): librados-devel-18.0.0-3124.gc19e8f2a. 2.3 MB/s | 139 kB 00:00 2023-03-24T22:22:15.793 INFO:teuthology.orchestra.run.smithi100.stdout:(31/142): rbd-mirror-18.0.0-3124.gc19e8f2a.el8. 4.1 MB/s | 3.4 MB 00:00 2023-03-24T22:22:15.818 INFO:teuthology.orchestra.run.smithi100.stdout:(32/142): ceph-mgr-modules-core-18.0.0-3124.gc1 3.1 MB/s | 239 kB 00:00 2023-03-24T22:22:15.852 INFO:teuthology.orchestra.run.smithi100.stdout:(33/142): ceph-mgr-rook-18.0.0-3124.gc19e8f2a.e 809 kB/s | 47 kB 00:00 2023-03-24T22:22:15.855 INFO:teuthology.orchestra.run.smithi018.stdout:(16/142): libradosstriper1-18.0.0-3124.gc19e8f2 4.1 MB/s | 474 kB 00:00 2023-03-24T22:22:15.877 INFO:teuthology.orchestra.run.smithi100.stdout:(34/142): ceph-prometheus-alerts-18.0.0-3124.gc 248 kB/s | 15 kB 00:00 2023-03-24T22:22:15.910 INFO:teuthology.orchestra.run.smithi100.stdout:(35/142): ceph-volume-18.0.0-3124.gc19e8f2a.el8 4.4 MB/s | 263 kB 00:00 2023-03-24T22:22:15.986 INFO:teuthology.orchestra.run.smithi100.stdout:(36/142): cephadm-18.0.0-3124.gc19e8f2a.el8.noa 764 kB/s | 83 kB 00:00 2023-03-24T22:22:16.011 INFO:teuthology.orchestra.run.smithi100.stdout:(37/142): cephfs-top-18.0.0-3124.gc19e8f2a.el8. 180 kB/s | 18 kB 00:00 2023-03-24T22:22:16.095 INFO:teuthology.orchestra.run.smithi100.stdout:(38/142): automake-1.16.1-7.el8.noarch.rpm 6.4 MB/s | 713 kB 00:00 2023-03-24T22:22:16.128 INFO:teuthology.orchestra.run.smithi100.stdout:(39/142): flex-2.6.1-9.el8.x86_64.rpm 9.4 MB/s | 320 kB 00:00 2023-03-24T22:22:16.154 INFO:teuthology.orchestra.run.smithi100.stdout:(40/142): bison-3.0.4-10.el8.x86_64.rpm 4.7 MB/s | 688 kB 00:00 2023-03-24T22:22:16.210 INFO:teuthology.orchestra.run.smithi100.stdout:(41/142): jq-1.6-6.el8.x86_64.rpm 2.4 MB/s | 202 kB 00:00 2023-03-24T22:22:16.246 INFO:teuthology.orchestra.run.smithi100.stdout:(42/142): libpq-13.5-1.el8.x86_64.rpm 2.1 MB/s | 198 kB 00:00 2023-03-24T22:22:16.271 INFO:teuthology.orchestra.run.smithi100.stdout:(43/142): librdkafka-0.11.4-3.el8.x86_64.rpm 5.7 MB/s | 354 kB 00:00 2023-03-24T22:22:16.305 INFO:teuthology.orchestra.run.smithi100.stdout:(44/142): lttng-ust-2.8.1-11.el8.x86_64.rpm 7.7 MB/s | 259 kB 00:00 2023-03-24T22:22:16.330 INFO:teuthology.orchestra.run.smithi100.stdout:(45/142): oniguruma-6.8.2-2.el8.x86_64.rpm 7.3 MB/s | 187 kB 00:00 2023-03-24T22:22:16.356 INFO:teuthology.orchestra.run.smithi100.stdout:(46/142): libtool-2.4.6-25.el8.x86_64.rpm 6.3 MB/s | 709 kB 00:00 2023-03-24T22:22:16.427 INFO:teuthology.orchestra.run.smithi018.stdout:(17/142): librgw2-18.0.0-3124.gc19e8f2a.el8.x86 11 MB/s | 6.5 MB 00:00 2023-03-24T22:22:16.469 INFO:teuthology.orchestra.run.smithi018.stdout:(18/142): python3-ceph-argparse-18.0.0-3124.gc1 1.0 MB/s | 45 kB 00:00 2023-03-24T22:22:16.514 INFO:teuthology.orchestra.run.smithi100.stdout:(47/142): openblas-0.3.15-4.el8.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2023-03-24T22:22:16.521 INFO:teuthology.orchestra.run.smithi018.stdout:(19/142): python3-ceph-common-18.0.0-3124.gc19e 2.3 MB/s | 117 kB 00:00 2023-03-24T22:22:16.577 INFO:teuthology.orchestra.run.smithi018.stdout:(20/142): python3-cephfs-18.0.0-3124.gc19e8f2a. 3.2 MB/s | 190 kB 00:00 2023-03-24T22:22:16.615 INFO:teuthology.orchestra.run.smithi100.stdout:(48/142): ceph-mgr-diskprediction-local-18.0.0- 7.0 MB/s | 7.4 MB 00:01 2023-03-24T22:22:16.665 INFO:teuthology.orchestra.run.smithi100.stdout:(49/142): postgresql-10.23-1.module_el8+300+2c5 10 MB/s | 1.6 MB 00:00 2023-03-24T22:22:16.669 INFO:teuthology.orchestra.run.smithi018.stdout:(21/142): python3-rados-18.0.0-3124.gc19e8f2a.e 3.9 MB/s | 370 kB 00:00 2023-03-24T22:22:16.728 INFO:teuthology.orchestra.run.smithi018.stdout:(22/142): python3-rbd-18.0.0-3124.gc19e8f2a.el8 6.1 MB/s | 364 kB 00:00 2023-03-24T22:22:16.757 INFO:teuthology.orchestra.run.smithi100.stdout:(50/142): openblas-threads-0.3.15-4.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2023-03-24T22:22:16.770 INFO:teuthology.orchestra.run.smithi018.stdout:(23/142): python3-rgw-18.0.0-3124.gc19e8f2a.el8 2.5 MB/s | 107 kB 00:00 2023-03-24T22:22:16.820 INFO:teuthology.orchestra.run.smithi018.stdout:(24/142): rbd-fuse-18.0.0-3124.gc19e8f2a.el8.x8 1.8 MB/s | 94 kB 00:00 2023-03-24T22:22:16.841 INFO:teuthology.orchestra.run.smithi100.stdout:(51/142): postgresql-contrib-10.23-1.module_el8 3.9 MB/s | 903 kB 00:00 2023-03-24T22:22:16.916 INFO:teuthology.orchestra.run.smithi100.stdout:(52/142): postgresql-server-10.23-1.module_el8+ 21 MB/s | 5.1 MB 00:00 2023-03-24T22:22:16.966 INFO:teuthology.orchestra.run.smithi100.stdout:(53/142): python3-jsonpatch-1.21-2.el8.noarch.r 531 kB/s | 27 kB 00:00 2023-03-24T22:22:16.992 INFO:teuthology.orchestra.run.smithi100.stdout:(54/142): python3-jinja2-2.10.1-3.el8.noarch.rp 3.5 MB/s | 538 kB 00:00 2023-03-24T22:22:17.017 INFO:teuthology.orchestra.run.smithi100.stdout:(55/142): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-03-24T22:22:17.042 INFO:teuthology.orchestra.run.smithi100.stdout:(56/142): python3-mako-1.0.6-14.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-03-24T22:22:17.067 INFO:teuthology.orchestra.run.smithi100.stdout:(57/142): python3-markupsafe-0.23-19.el8.x86_64 771 kB/s | 39 kB 00:00 2023-03-24T22:22:17.087 INFO:teuthology.orchestra.run.smithi018.stdout:(25/142): rbd-mirror-18.0.0-3124.gc19e8f2a.el8. 13 MB/s | 3.4 MB 00:00 2023-03-24T22:22:17.101 INFO:teuthology.orchestra.run.smithi100.stdout:(58/142): python3-numpy-f2py-1.14.3-10.el8.x86_ 6.6 MB/s | 225 kB 00:00 2023-03-24T22:22:17.126 INFO:teuthology.orchestra.run.smithi100.stdout:(59/142): python3-prettytable-0.7.2-14.el8.noar 1.7 MB/s | 44 kB 00:00 2023-03-24T22:22:17.146 INFO:teuthology.orchestra.run.smithi018.stdout:(26/142): rbd-nbd-18.0.0-3124.gc19e8f2a.el8.x86 3.0 MB/s | 179 kB 00:00 2023-03-24T22:22:17.151 INFO:teuthology.orchestra.run.smithi100.stdout:(60/142): python3-pyOpenSSL-19.0.0-1.el8.noarch 4.0 MB/s | 103 kB 00:00 2023-03-24T22:22:17.196 INFO:teuthology.orchestra.run.smithi018.stdout:(27/142): ceph-grafana-dashboards-18.0.0-3124.g 461 kB/s | 23 kB 00:00 2023-03-24T22:22:17.197 INFO:teuthology.orchestra.run.smithi100.stdout:(61/142): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.7 MB/s | 126 kB 00:00 2023-03-24T22:22:17.236 INFO:teuthology.orchestra.run.smithi100.stdout:(62/142): python3-pyasn1-modules-0.3.7-6.el8.no 2.9 MB/s | 110 kB 00:00 2023-03-24T22:22:17.246 INFO:teuthology.orchestra.run.smithi018.stdout:(28/142): ceph-mgr-cephadm-18.0.0-3124.gc19e8f2 2.4 MB/s | 123 kB 00:00 2023-03-24T22:22:17.294 INFO:teuthology.orchestra.run.smithi100.stdout:(63/142): python3-babel-2.5.1-7.el8.noarch.rpm 8.9 MB/s | 4.8 MB 00:00 2023-03-24T22:22:17.319 INFO:teuthology.orchestra.run.smithi100.stdout:(64/142): python3-pytz-2017.2-9.el8.noarch.rpm 647 kB/s | 54 kB 00:00 2023-03-24T22:22:17.353 INFO:teuthology.orchestra.run.smithi100.stdout:(65/142): python3-werkzeug-0.12.2-4.el8.noarch. 13 MB/s | 457 kB 00:00 2023-03-24T22:22:17.387 INFO:teuthology.orchestra.run.smithi100.stdout:(66/142): socat-1.7.4.1-1.el8.x86_64.rpm 9.4 MB/s | 323 kB 00:00 2023-03-24T22:22:17.424 INFO:teuthology.orchestra.run.smithi018.stdout:(29/142): ceph-mgr-dashboard-18.0.0-3124.gc19e8 9.6 MB/s | 1.7 MB 00:00 2023-03-24T22:22:17.470 INFO:teuthology.orchestra.run.smithi100.stdout:(67/142): uuid-1.6.2-43.el8.x86_64.rpm 767 kB/s | 64 kB 00:00 2023-03-24T22:22:17.496 INFO:teuthology.orchestra.run.smithi100.stdout:(68/142): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 71 kB 00:00 2023-03-24T22:22:17.563 INFO:teuthology.orchestra.run.smithi100.stdout:(69/142): python3-numpy-1.14.3-10.el8.x86_64.rp 7.2 MB/s | 3.7 MB 00:00 2023-03-24T22:22:17.588 INFO:teuthology.orchestra.run.smithi100.stdout:(70/142): elfutils-libelf-devel-0.188-3.el8.x86 666 kB/s | 61 kB 00:00 2023-03-24T22:22:17.605 INFO:teuthology.orchestra.run.smithi100.stdout:(71/142): fuse-2.9.7-16.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-03-24T22:22:17.630 INFO:teuthology.orchestra.run.smithi100.stdout:(72/142): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-03-24T22:22:17.656 INFO:teuthology.orchestra.run.smithi100.stdout:(73/142): libaio-devel-0.3.112-1.el8.x86_64.rpm 746 kB/s | 19 kB 00:00 2023-03-24T22:22:17.704 INFO:teuthology.orchestra.run.smithi100.stdout:(74/142): libcom_err-devel-1.45.6-5.el8.x86_64. 805 kB/s | 39 kB 00:00 2023-03-24T22:22:17.740 INFO:teuthology.orchestra.run.smithi100.stdout:(75/142): krb5-devel-1.18.2-22.el8.x86_64.rpm 4.1 MB/s | 561 kB 00:00 2023-03-24T22:22:17.773 INFO:teuthology.orchestra.run.smithi100.stdout:(76/142): libquadmath-8.5.0-18.el8.x86_64.rpm 5.0 MB/s | 172 kB 00:00 2023-03-24T22:22:17.798 INFO:teuthology.orchestra.run.smithi100.stdout:(77/142): libgfortran-8.5.0-18.el8.x86_64.rpm 6.7 MB/s | 645 kB 00:00 2023-03-24T22:22:17.824 INFO:teuthology.orchestra.run.smithi100.stdout:(78/142): librabbitmq-0.9.0-3.el8.x86_64.rpm 935 kB/s | 47 kB 00:00 2023-03-24T22:22:17.858 INFO:teuthology.orchestra.run.smithi100.stdout:(79/142): libselinux-devel-2.9-6.el8.x86_64.rpm 3.3 MB/s | 200 kB 00:00 2023-03-24T22:22:17.883 INFO:teuthology.orchestra.run.smithi100.stdout:(80/142): libsepol-devel-2.9-3.el8.x86_64.rpm 1.4 MB/s | 87 kB 00:00 2023-03-24T22:22:17.908 INFO:teuthology.orchestra.run.smithi100.stdout:(81/142): libuuid-devel-2.32.1-41.el8.x86_64.rp 1.9 MB/s | 99 kB 00:00 2023-03-24T22:22:17.957 INFO:teuthology.orchestra.run.smithi100.stdout:(82/142): libverto-devel-0.3.2-2.el8.x86_64.rpm 243 kB/s | 18 kB 00:00 2023-03-24T22:22:18.026 INFO:teuthology.orchestra.run.smithi100.stdout:(83/142): pcre2-devel-10.32-3.el8.x86_64.rpm 8.7 MB/s | 605 kB 00:00 2023-03-24T22:22:18.031 INFO:teuthology.orchestra.run.smithi018.stdout:(30/142): ceph-osd-18.0.0-3124.gc19e8f2a.el8.x8 3.8 MB/s | 18 MB 00:04 2023-03-24T22:22:18.067 INFO:teuthology.orchestra.run.smithi100.stdout:(84/142): pcre2-utf16-10.32-3.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-03-24T22:22:18.109 INFO:teuthology.orchestra.run.smithi100.stdout:(85/142): pcre2-utf32-10.32-3.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-03-24T22:22:18.131 INFO:teuthology.orchestra.run.smithi018.stdout:(31/142): ceph-mgr-diskprediction-local-18.0.0- 10 MB/s | 7.4 MB 00:00 2023-03-24T22:22:18.156 INFO:teuthology.orchestra.run.smithi018.stdout:(32/142): ceph-mgr-modules-core-18.0.0-3124.gc1 1.9 MB/s | 239 kB 00:00 2023-03-24T22:22:18.159 INFO:teuthology.orchestra.run.smithi100.stdout:(86/142): openssl-devel-1.1.1k-9.el8.x86_64.rpm 13 MB/s | 3.2 MB 00:00 2023-03-24T22:22:18.181 INFO:teuthology.orchestra.run.smithi018.stdout:(33/142): ceph-mgr-rook-18.0.0-3124.gc19e8f2a.e 948 kB/s | 47 kB 00:00 2023-03-24T22:22:18.218 INFO:teuthology.orchestra.run.smithi100.stdout:(87/142): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 237 kB 00:00 2023-03-24T22:22:18.231 INFO:teuthology.orchestra.run.smithi018.stdout:(34/142): ceph-prometheus-alerts-18.0.0-3124.gc 193 kB/s | 15 kB 00:00 2023-03-24T22:22:18.235 INFO:teuthology.orchestra.run.smithi100.stdout:(88/142): python3-jwt-1.6.1-2.el8.noarch.rpm 2.5 MB/s | 43 kB 00:00 2023-03-24T22:22:18.261 INFO:teuthology.orchestra.run.smithi100.stdout:(89/142): python3-cryptography-3.2.1-5.el8.x86_ 5.4 MB/s | 559 kB 00:00 2023-03-24T22:22:18.282 INFO:teuthology.orchestra.run.smithi018.stdout:(35/142): ceph-volume-18.0.0-3124.gc19e8f2a.el8 2.6 MB/s | 263 kB 00:00 2023-03-24T22:22:18.286 INFO:teuthology.orchestra.run.smithi100.stdout:(90/142): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-03-24T22:22:18.307 INFO:teuthology.orchestra.run.smithi018.stdout:(36/142): cephadm-18.0.0-3124.gc19e8f2a.el8.noa 1.1 MB/s | 83 kB 00:00 2023-03-24T22:22:18.328 INFO:teuthology.orchestra.run.smithi100.stdout:(91/142): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-03-24T22:22:18.332 INFO:teuthology.orchestra.run.smithi018.stdout:(37/142): cephfs-top-18.0.0-3124.gc19e8f2a.el8. 358 kB/s | 18 kB 00:00 2023-03-24T22:22:18.353 INFO:teuthology.orchestra.run.smithi100.stdout:(92/142): python3-requests-oauthlib-1.0.0-1.el8 639 kB/s | 43 kB 00:00 2023-03-24T22:22:18.378 INFO:teuthology.orchestra.run.smithi100.stdout:(93/142): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-03-24T22:22:18.403 INFO:teuthology.orchestra.run.smithi100.stdout:(94/142): zlib-devel-1.2.11-19.el8.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-03-24T22:22:18.408 INFO:teuthology.orchestra.run.smithi018.stdout:(38/142): automake-1.16.1-7.el8.noarch.rpm 7.0 MB/s | 713 kB 00:00 2023-03-24T22:22:18.441 INFO:teuthology.orchestra.run.smithi018.stdout:(39/142): flex-2.6.1-9.el8.x86_64.rpm 9.4 MB/s | 320 kB 00:00 2023-03-24T22:22:18.467 INFO:teuthology.orchestra.run.smithi018.stdout:(40/142): bison-3.0.4-10.el8.x86_64.rpm 5.0 MB/s | 688 kB 00:00 2023-03-24T22:22:18.492 INFO:teuthology.orchestra.run.smithi018.stdout:(41/142): jq-1.6-6.el8.x86_64.rpm 3.9 MB/s | 202 kB 00:00 2023-03-24T22:22:18.526 INFO:teuthology.orchestra.run.smithi018.stdout:(42/142): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-03-24T22:22:18.562 INFO:teuthology.orchestra.run.smithi100.stdout:(95/142): python3-scipy-1.0.0-21.module_el8.5.0 11 MB/s | 14 MB 00:01 2023-03-24T22:22:18.587 INFO:teuthology.orchestra.run.smithi018.stdout:(43/142): librdkafka-0.11.4-3.el8.x86_64.rpm 3.7 MB/s | 354 kB 00:00 2023-03-24T22:22:18.643 INFO:teuthology.orchestra.run.smithi018.stdout:(44/142): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.5 MB/s | 259 kB 00:00 2023-03-24T22:22:18.669 INFO:teuthology.orchestra.run.smithi018.stdout:(45/142): oniguruma-6.8.2-2.el8.x86_64.rpm 7.4 MB/s | 187 kB 00:00 2023-03-24T22:22:18.702 INFO:teuthology.orchestra.run.smithi018.stdout:(46/142): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2023-03-24T22:22:18.709 INFO:teuthology.orchestra.run.smithi100.stdout:(96/142): liboath-2.6.2-3.el8.x86_64.rpm 193 kB/s | 59 kB 00:00 2023-03-24T22:22:18.738 INFO:teuthology.orchestra.run.smithi100.stdout:(97/142): libunwind-1.3.1-3.el8.x86_64.rpm 428 kB/s | 75 kB 00:00 2023-03-24T22:22:18.805 INFO:teuthology.orchestra.run.smithi100.stdout:(98/142): gperftools-libs-2.7-9.el8.x86_64.rpm 718 kB/s | 306 kB 00:00 2023-03-24T22:22:18.819 INFO:teuthology.orchestra.run.smithi018.stdout:(47/142): openblas-0.3.15-4.el8.x86_64.rpm 31 MB/s | 4.7 MB 00:00 2023-03-24T22:22:18.863 INFO:teuthology.orchestra.run.smithi100.stdout:(99/142): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 348 kB/s | 44 kB 00:00 2023-03-24T22:22:18.870 INFO:teuthology.orchestra.run.smithi018.stdout:(48/142): postgresql-10.23-1.module_el8+300+2c5 31 MB/s | 1.6 MB 00:00 2023-03-24T22:22:18.889 INFO:teuthology.orchestra.run.smithi100.stdout:(100/142): python3-beautifulsoup4-4.6.3-2.el8.1 2.2 MB/s | 185 kB 00:00 2023-03-24T22:22:18.920 INFO:teuthology.orchestra.run.smithi018.stdout:(49/142): postgresql-contrib-10.23-1.module_el8 18 MB/s | 903 kB 00:00 2023-03-24T22:22:18.922 INFO:teuthology.orchestra.run.smithi100.stdout:(101/142): python3-asyncssh-2.7.0-2.el8.noarch. 2.2 MB/s | 476 kB 00:00 2023-03-24T22:22:18.973 INFO:teuthology.orchestra.run.smithi100.stdout:(102/142): python3-certifi-2018.10.15-7.el8.noa 194 kB/s | 16 kB 00:00 2023-03-24T22:22:18.998 INFO:teuthology.orchestra.run.smithi100.stdout:(103/142): python3-cachetools-3.1.1-4.el8.noarc 245 kB/s | 33 kB 00:00 2023-03-24T22:22:19.024 INFO:teuthology.orchestra.run.smithi100.stdout:(104/142): python3-cheroot-8.5.2-1.el8.noarch.r 1.7 MB/s | 173 kB 00:00 2023-03-24T22:22:19.049 INFO:teuthology.orchestra.run.smithi100.stdout:(105/142): python3-google-auth-1.1.1-10.el8.noa 1.6 MB/s | 81 kB 00:00 2023-03-24T22:22:19.062 INFO:teuthology.orchestra.run.smithi018.stdout:(50/142): postgresql-server-10.23-1.module_el8+ 36 MB/s | 5.1 MB 00:00 2023-03-24T22:22:19.074 INFO:teuthology.orchestra.run.smithi100.stdout:(106/142): python3-cherrypy-18.4.0-1.el8.noarch 3.7 MB/s | 384 kB 00:00 2023-03-24T22:22:19.099 INFO:teuthology.orchestra.run.smithi100.stdout:(107/142): python3-jaraco-6.2-6.el8.noarch.rpm 146 kB/s | 11 kB 00:00 2023-03-24T22:22:19.125 INFO:teuthology.orchestra.run.smithi100.stdout:(108/142): python3-jaraco-functools-2.0-4.el8.n 231 kB/s | 18 kB 00:00 2023-03-24T22:22:19.151 INFO:teuthology.orchestra.run.smithi100.stdout:(109/142): python3-logutils-0.3.5-11.el8.noarch 966 kB/s | 49 kB 00:00 2023-03-24T22:22:19.187 INFO:teuthology.orchestra.run.smithi018.stdout:(51/142): python3-babel-2.5.1-7.el8.noarch.rpm 38 MB/s | 4.8 MB 00:00 2023-03-24T22:22:19.199 INFO:teuthology.orchestra.run.smithi100.stdout:(110/142): python3-more-itertools-7.2.0-3.el8.n 790 kB/s | 59 kB 00:00 2023-03-24T22:22:19.234 INFO:teuthology.orchestra.run.smithi100.stdout:(111/142): python3-natsort-7.1.1-2.el8.noarch.r 722 kB/s | 60 kB 00:00 2023-03-24T22:22:19.263 INFO:teuthology.orchestra.run.smithi018.stdout:(52/142): openblas-threads-0.3.15-4.el8.x86_64. 8.7 MB/s | 4.9 MB 00:00 2023-03-24T22:22:19.267 INFO:teuthology.orchestra.run.smithi100.stdout:(112/142): python3-kubernetes-11.0.0-6.el8.noar 6.0 MB/s | 1.2 MB 00:00 2023-03-24T22:22:19.288 INFO:teuthology.orchestra.run.smithi018.stdout:(53/142): python3-jinja2-2.10.1-3.el8.noarch.rp 5.2 MB/s | 538 kB 00:00 2023-03-24T22:22:19.293 INFO:teuthology.orchestra.run.smithi100.stdout:(113/142): python3-portend-2.6-1.el8.noarch.rpm 265 kB/s | 16 kB 00:00 2023-03-24T22:22:19.313 INFO:teuthology.orchestra.run.smithi018.stdout:(54/142): python3-jsonpatch-1.21-2.el8.noarch.r 531 kB/s | 27 kB 00:00 2023-03-24T22:22:19.318 INFO:teuthology.orchestra.run.smithi100.stdout:(114/142): python3-repoze-lru-0.7-6.el8.noarch. 669 kB/s | 34 kB 00:00 2023-03-24T22:22:19.338 INFO:teuthology.orchestra.run.smithi018.stdout:(55/142): python3-jsonpointer-1.10-11.el8.noarc 397 kB/s | 20 kB 00:00 2023-03-24T22:22:19.343 INFO:teuthology.orchestra.run.smithi100.stdout:(115/142): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-03-24T22:22:19.364 INFO:teuthology.orchestra.run.smithi018.stdout:(56/142): python3-mako-1.0.6-14.el8.noarch.rpm 3.0 MB/s | 157 kB 00:00 2023-03-24T22:22:19.369 INFO:teuthology.orchestra.run.smithi100.stdout:(116/142): python3-routes-2.4.1-12.el8.noarch.r 2.5 MB/s | 196 kB 00:00 2023-03-24T22:22:19.394 INFO:teuthology.orchestra.run.smithi100.stdout:(117/142): python3-rsa-4.8-1.el8.noarch.rpm 813 kB/s | 61 kB 00:00 2023-03-24T22:22:19.414 INFO:teuthology.orchestra.run.smithi018.stdout:(57/142): python3-markupsafe-0.23-19.el8.x86_64 515 kB/s | 39 kB 00:00 2023-03-24T22:22:19.420 INFO:teuthology.orchestra.run.smithi100.stdout:(118/142): python3-simplegeneric-0.8.1-17.el8.n 247 kB/s | 19 kB 00:00 2023-03-24T22:22:19.439 INFO:teuthology.orchestra.run.smithi018.stdout:(58/142): python3-numpy-f2py-1.14.3-10.el8.x86_ 8.8 MB/s | 225 kB 00:00 2023-03-24T22:22:19.462 INFO:teuthology.orchestra.run.smithi100.stdout:(119/142): python3-singledispatch-3.4.0.3-18.el 259 kB/s | 24 kB 00:00 2023-03-24T22:22:19.464 INFO:teuthology.orchestra.run.smithi018.stdout:(59/142): python3-prettytable-0.7.2-14.el8.noar 1.7 MB/s | 44 kB 00:00 2023-03-24T22:22:19.486 INFO:teuthology.orchestra.run.smithi100.stdout:(120/142): python3-tempora-1.14.1-5.el8.noarch. 317 kB/s | 29 kB 00:00 2023-03-24T22:22:19.490 INFO:teuthology.orchestra.run.smithi018.stdout:(60/142): python3-pyOpenSSL-19.0.0-1.el8.noarch 4.1 MB/s | 103 kB 00:00 2023-03-24T22:22:19.503 INFO:teuthology.orchestra.run.smithi100.stdout:(121/142): python3-trustme-0.6.0-4.el8.noarch.r 329 kB/s | 27 kB 00:00 2023-03-24T22:22:19.515 INFO:teuthology.orchestra.run.smithi018.stdout:(61/142): python3-pyasn1-0.3.7-6.el8.noarch.rpm 5.0 MB/s | 126 kB 00:00 2023-03-24T22:22:19.529 INFO:teuthology.orchestra.run.smithi100.stdout:(122/142): python3-waitress-1.2.1-2.el8.1.noarc 3.5 MB/s | 241 kB 00:00 2023-03-24T22:22:19.540 INFO:teuthology.orchestra.run.smithi018.stdout:(62/142): python3-pyasn1-modules-0.3.7-6.el8.no 4.3 MB/s | 110 kB 00:00 2023-03-24T22:22:19.554 INFO:teuthology.orchestra.run.smithi100.stdout:(123/142): python3-websocket-client-0.56.0-5.el 1.2 MB/s | 61 kB 00:00 2023-03-24T22:22:19.565 INFO:teuthology.orchestra.run.smithi018.stdout:(63/142): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-03-24T22:22:19.579 INFO:teuthology.orchestra.run.smithi100.stdout:(124/142): python3-webob-1.8.5-1.el8.1.noarch.r 3.2 MB/s | 251 kB 00:00 2023-03-24T22:22:19.605 INFO:teuthology.orchestra.run.smithi100.stdout:(125/142): python3-webtest-2.0.33-1.el8.noarch. 1.1 MB/s | 85 kB 00:00 2023-03-24T22:22:19.622 INFO:teuthology.orchestra.run.smithi100.stdout:(126/142): python3-zc-lockfile-2.0-2.el8.noarch 342 kB/s | 23 kB 00:00 2023-03-24T22:22:19.766 INFO:teuthology.orchestra.run.smithi018.stdout:(64/142): python3-numpy-1.14.3-10.el8.x86_64.rp 9.3 MB/s | 3.7 MB 00:00 2023-03-24T22:22:19.805 INFO:teuthology.orchestra.run.smithi100.stdout:(127/142): thrift-0.13.0-2.el8.x86_64.rpm 7.7 MB/s | 1.7 MB 00:00 2023-03-24T22:22:19.824 INFO:teuthology.orchestra.run.smithi018.stdout:(65/142): python3-werkzeug-0.12.2-4.el8.noarch. 7.7 MB/s | 457 kB 00:00 2023-03-24T22:22:19.831 INFO:teuthology.orchestra.run.smithi100.stdout:(128/142): NetworkManager-initscripts-updown-1. 5.6 MB/s | 142 kB 00:00 2023-03-24T22:22:19.856 INFO:teuthology.orchestra.run.smithi100.stdout:(129/142): elfutils-debuginfod-client-0.188-3.e 3.0 MB/s | 75 kB 00:00 2023-03-24T22:22:19.881 INFO:teuthology.orchestra.run.smithi100.stdout:(130/142): elfutils-libelf-0.188-3.el8.x86_64.r 9.1 MB/s | 231 kB 00:00 2023-03-24T22:22:19.906 INFO:teuthology.orchestra.run.smithi100.stdout:(131/142): elfutils-libs-0.188-3.el8.x86_64.rpm 12 MB/s | 299 kB 00:00 2023-03-24T22:22:19.965 INFO:teuthology.orchestra.run.smithi100.stdout:(132/142): libblkid-2.32.1-41.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-03-24T22:22:19.975 INFO:teuthology.orchestra.run.smithi018.stdout:(66/142): python3-scipy-1.0.0-21.module_el8.5.0 33 MB/s | 14 MB 00:00 2023-03-24T22:22:19.991 INFO:teuthology.orchestra.run.smithi100.stdout:(133/142): libfdisk-2.32.1-41.el8.x86_64.rpm 9.9 MB/s | 253 kB 00:00 2023-03-24T22:22:20.000 INFO:teuthology.orchestra.run.smithi018.stdout:(67/142): uuid-1.6.2-43.el8.x86_64.rpm 2.5 MB/s | 64 kB 00:00 2023-03-24T22:22:20.016 INFO:teuthology.orchestra.run.smithi100.stdout:(134/142): libmount-2.32.1-41.el8.x86_64.rpm 9.2 MB/s | 236 kB 00:00 2023-03-24T22:22:20.025 INFO:teuthology.orchestra.run.smithi018.stdout:(68/142): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2023-03-24T22:22:20.051 INFO:teuthology.orchestra.run.smithi018.stdout:(69/142): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2023-03-24T22:22:20.075 INFO:teuthology.orchestra.run.smithi100.stdout:(135/142): librbd1-18.0.0-3124.gc19e8f2a.el8.x8 7.9 MB/s | 3.6 MB 00:00 2023-03-24T22:22:20.076 INFO:teuthology.orchestra.run.smithi018.stdout:(70/142): elfutils-libelf-devel-0.188-3.el8.x86 1.2 MB/s | 61 kB 00:00 2023-03-24T22:22:20.100 INFO:teuthology.orchestra.run.smithi100.stdout:(136/142): libsmartcols-2.32.1-41.el8.x86_64.rp 2.1 MB/s | 179 kB 00:00 2023-03-24T22:22:20.101 INFO:teuthology.orchestra.run.smithi018.stdout:(71/142): fuse-2.9.7-16.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-03-24T22:22:20.125 INFO:teuthology.orchestra.run.smithi100.stdout:(137/142): libuuid-2.32.1-41.el8.x86_64.rpm 1.9 MB/s | 98 kB 00:00 2023-03-24T22:22:20.127 INFO:teuthology.orchestra.run.smithi018.stdout:(72/142): keyutils-libs-devel-1.5.10-9.el8.x86_ 956 kB/s | 48 kB 00:00 2023-03-24T22:22:20.152 INFO:teuthology.orchestra.run.smithi018.stdout:(73/142): libaio-devel-0.3.112-1.el8.x86_64.rpm 752 kB/s | 19 kB 00:00 2023-03-24T22:22:20.176 INFO:teuthology.orchestra.run.smithi100.stdout:(138/142): librados2-18.0.0-3124.gc19e8f2a.el8. 6.4 MB/s | 3.7 MB 00:00 2023-03-24T22:22:20.196 INFO:teuthology.orchestra.run.smithi018.stdout:(74/142): libcom_err-devel-1.45.6-5.el8.x86_64. 890 kB/s | 39 kB 00:00 2023-03-24T22:22:20.228 INFO:teuthology.orchestra.run.smithi018.stdout:(75/142): krb5-devel-1.18.2-22.el8.x86_64.rpm 4.3 MB/s | 561 kB 00:00 2023-03-24T22:22:20.234 INFO:teuthology.orchestra.run.smithi100.stdout:(139/142): openssl-1.1.1k-9.el8.x86_64.rpm 5.4 MB/s | 737 kB 00:00 2023-03-24T22:22:20.253 INFO:teuthology.orchestra.run.smithi018.stdout:(76/142): libquadmath-8.5.0-18.el8.x86_64.rpm 6.8 MB/s | 172 kB 00:00 2023-03-24T22:22:20.259 INFO:teuthology.orchestra.run.smithi100.stdout:(140/142): util-linux-user-2.32.1-41.el8.x86_64 4.0 MB/s | 102 kB 00:00 2023-03-24T22:22:20.278 INFO:teuthology.orchestra.run.smithi018.stdout:(77/142): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-03-24T22:22:20.301 INFO:teuthology.orchestra.run.smithi100.stdout:(141/142): openssl-libs-1.1.1k-9.el8.x86_64.rpm 8.4 MB/s | 1.5 MB 00:00 2023-03-24T22:22:20.303 INFO:teuthology.orchestra.run.smithi018.stdout:(78/142): libgfortran-8.5.0-18.el8.x86_64.rpm 5.9 MB/s | 645 kB 00:00 2023-03-24T22:22:20.329 INFO:teuthology.orchestra.run.smithi018.stdout:(79/142): libselinux-devel-2.9-6.el8.x86_64.rpm 3.9 MB/s | 200 kB 00:00 2023-03-24T22:22:20.354 INFO:teuthology.orchestra.run.smithi018.stdout:(80/142): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-03-24T22:22:20.379 INFO:teuthology.orchestra.run.smithi018.stdout:(81/142): libuuid-devel-2.32.1-41.el8.x86_64.rp 1.9 MB/s | 99 kB 00:00 2023-03-24T22:22:20.385 INFO:teuthology.orchestra.run.smithi100.stdout:(142/142): util-linux-2.32.1-41.el8.x86_64.rpm 12 MB/s | 2.6 MB 00:00 2023-03-24T22:22:20.390 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2023-03-24T22:22:20.390 INFO:teuthology.orchestra.run.smithi100.stdout:Total 18 MB/s | 204 MB 00:11 2023-03-24T22:22:20.396 INFO:teuthology.orchestra.run.smithi018.stdout:(82/142): libverto-devel-0.3.2-2.el8.x86_64.rpm 431 kB/s | 18 kB 00:00 2023-03-24T22:22:20.438 INFO:teuthology.orchestra.run.smithi018.stdout:(83/142): pcre2-devel-10.32-3.el8.x86_64.rpm 14 MB/s | 605 kB 00:00 2023-03-24T22:22:20.472 INFO:teuthology.orchestra.run.smithi018.stdout:(84/142): pcre2-utf16-10.32-3.el8.x86_64.rpm 6.7 MB/s | 229 kB 00:00 2023-03-24T22:22:20.505 INFO:teuthology.orchestra.run.smithi018.stdout:(85/142): pcre2-utf32-10.32-3.el8.x86_64.rpm 6.5 MB/s | 220 kB 00:00 2023-03-24T22:22:20.539 INFO:teuthology.orchestra.run.smithi018.stdout:(86/142): python3-cffi-1.11.5-5.el8.x86_64.rpm 6.9 MB/s | 237 kB 00:00 2023-03-24T22:22:20.623 INFO:teuthology.orchestra.run.smithi018.stdout:(87/142): openssl-devel-1.1.1k-9.el8.x86_64.rpm 13 MB/s | 3.2 MB 00:00 2023-03-24T22:22:20.648 INFO:teuthology.orchestra.run.smithi018.stdout:(88/142): python3-jwt-1.6.1-2.el8.noarch.rpm 1.7 MB/s | 43 kB 00:00 2023-03-24T22:22:20.682 INFO:teuthology.orchestra.run.smithi018.stdout:(89/142): python3-cryptography-3.2.1-5.el8.x86_ 3.8 MB/s | 559 kB 00:00 2023-03-24T22:22:20.707 INFO:teuthology.orchestra.run.smithi018.stdout:(90/142): python3-oauthlib-2.1.0-1.el8.noarch.r 2.6 MB/s | 155 kB 00:00 2023-03-24T22:22:20.732 INFO:teuthology.orchestra.run.smithi018.stdout:(91/142): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-03-24T22:22:20.757 INFO:teuthology.orchestra.run.smithi018.stdout:(92/142): python3-requests-oauthlib-1.0.0-1.el8 849 kB/s | 43 kB 00:00 2023-03-24T22:22:20.783 INFO:teuthology.orchestra.run.smithi018.stdout:(93/142): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-03-24T22:22:20.808 INFO:teuthology.orchestra.run.smithi018.stdout:(94/142): zlib-devel-1.2.11-19.el8.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-03-24T22:22:20.942 INFO:teuthology.orchestra.run.smithi018.stdout:(95/142): liboath-2.6.2-3.el8.x86_64.rpm 441 kB/s | 59 kB 00:00 2023-03-24T22:22:21.025 INFO:teuthology.orchestra.run.smithi018.stdout:(96/142): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-03-24T22:22:21.059 INFO:teuthology.orchestra.run.smithi018.stdout:(97/142): libunwind-1.3.1-3.el8.x86_64.rpm 641 kB/s | 75 kB 00:00 2023-03-24T22:22:21.110 INFO:teuthology.orchestra.run.smithi018.stdout:(98/142): python3-asyncssh-2.7.0-2.el8.noarch.r 5.5 MB/s | 476 kB 00:00 2023-03-24T22:22:21.135 INFO:teuthology.orchestra.run.smithi018.stdout:(99/142): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 580 kB/s | 44 kB 00:00 2023-03-24T22:22:21.160 INFO:teuthology.orchestra.run.smithi018.stdout:(100/142): python3-beautifulsoup4-4.6.3-2.el8.1 3.6 MB/s | 185 kB 00:00 2023-03-24T22:22:21.186 INFO:teuthology.orchestra.run.smithi018.stdout:(101/142): python3-cachetools-3.1.1-4.el8.noarc 654 kB/s | 33 kB 00:00 2023-03-24T22:22:21.211 INFO:teuthology.orchestra.run.smithi018.stdout:(102/142): python3-certifi-2018.10.15-7.el8.noa 324 kB/s | 16 kB 00:00 2023-03-24T22:22:21.269 INFO:teuthology.orchestra.run.smithi018.stdout:(103/142): python3-cheroot-8.5.2-1.el8.noarch.r 2.0 MB/s | 173 kB 00:00 2023-03-24T22:22:21.293 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2023-03-24T22:22:21.295 INFO:teuthology.orchestra.run.smithi018.stdout:(104/142): python3-cherrypy-18.4.0-1.el8.noarch 4.5 MB/s | 384 kB 00:00 2023-03-24T22:22:21.556 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2023-03-24T22:22:21.557 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2023-03-24T22:22:21.654 INFO:teuthology.orchestra.run.smithi018.stdout:(105/142): ceph-test-18.0.0-3124.gc19e8f2a.el8. 6.3 MB/s | 41 MB 00:06 2023-03-24T22:22:21.679 INFO:teuthology.orchestra.run.smithi018.stdout:(106/142): python3-jaraco-6.2-6.el8.noarch.rpm 29 kB/s | 11 kB 00:00 2023-03-24T22:22:21.712 INFO:teuthology.orchestra.run.smithi018.stdout:(107/142): python3-google-auth-1.1.1-10.el8.noa 183 kB/s | 81 kB 00:00 2023-03-24T22:22:21.738 INFO:teuthology.orchestra.run.smithi018.stdout:(108/142): python3-jaraco-functools-2.0-4.el8.n 210 kB/s | 18 kB 00:00 2023-03-24T22:22:21.785 INFO:teuthology.orchestra.run.smithi018.stdout:(109/142): python3-logutils-0.3.5-11.el8.noarch 673 kB/s | 49 kB 00:00 2023-03-24T22:22:21.813 INFO:teuthology.orchestra.run.smithi018.stdout:(110/142): python3-more-itertools-7.2.0-3.el8.n 779 kB/s | 59 kB 00:00 2023-03-24T22:22:21.839 INFO:teuthology.orchestra.run.smithi018.stdout:(111/142): python3-natsort-7.1.1-2.el8.noarch.r 1.1 MB/s | 60 kB 00:00 2023-03-24T22:22:21.897 INFO:teuthology.orchestra.run.smithi018.stdout:(112/142): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-03-24T22:22:21.923 INFO:teuthology.orchestra.run.smithi018.stdout:(113/142): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-03-24T22:22:21.948 INFO:teuthology.orchestra.run.smithi018.stdout:(114/142): python3-repoze-lru-0.7-6.el8.noarch. 669 kB/s | 34 kB 00:00 2023-03-24T22:22:21.982 INFO:teuthology.orchestra.run.smithi018.stdout:(115/142): python3-routes-2.4.1-12.el8.noarch.r 3.3 MB/s | 196 kB 00:00 2023-03-24T22:22:22.007 INFO:teuthology.orchestra.run.smithi018.stdout:(116/142): python3-rsa-4.8-1.el8.noarch.rpm 1.0 MB/s | 61 kB 00:00 2023-03-24T22:22:22.040 INFO:teuthology.orchestra.run.smithi018.stdout:(117/142): python3-kubernetes-11.0.0-6.el8.noar 3.2 MB/s | 1.2 MB 00:00 2023-03-24T22:22:22.057 INFO:teuthology.orchestra.run.smithi018.stdout:(118/142): python3-simplegeneric-0.8.1-17.el8.n 248 kB/s | 19 kB 00:00 2023-03-24T22:22:22.074 INFO:teuthology.orchestra.run.smithi018.stdout:(119/142): python3-singledispatch-3.4.0.3-18.el 357 kB/s | 24 kB 00:00 2023-03-24T22:22:22.099 INFO:teuthology.orchestra.run.smithi018.stdout:(120/142): python3-tempora-1.14.1-5.el8.noarch. 497 kB/s | 29 kB 00:00 2023-03-24T22:22:22.125 INFO:teuthology.orchestra.run.smithi018.stdout:(121/142): python3-trustme-0.6.0-4.el8.noarch.r 548 kB/s | 27 kB 00:00 2023-03-24T22:22:22.150 INFO:teuthology.orchestra.run.smithi018.stdout:(122/142): python3-webob-1.8.5-1.el8.1.noarch.r 4.9 MB/s | 251 kB 00:00 2023-03-24T22:22:22.201 INFO:teuthology.orchestra.run.smithi018.stdout:(123/142): python3-waitress-1.2.1-2.el8.1.noarc 1.9 MB/s | 241 kB 00:00 2023-03-24T22:22:22.226 INFO:teuthology.orchestra.run.smithi018.stdout:(124/142): python3-websocket-client-0.56.0-5.el 607 kB/s | 61 kB 00:00 2023-03-24T22:22:22.243 INFO:teuthology.orchestra.run.smithi018.stdout:(125/142): python3-webtest-2.0.33-1.el8.noarch. 923 kB/s | 85 kB 00:00 2023-03-24T22:22:22.260 INFO:teuthology.orchestra.run.smithi018.stdout:(126/142): python3-zc-lockfile-2.0-2.el8.noarch 389 kB/s | 23 kB 00:00 2023-03-24T22:22:22.502 INFO:teuthology.orchestra.run.smithi018.stdout:(127/142): thrift-0.13.0-2.el8.x86_64.rpm 6.3 MB/s | 1.7 MB 00:00 2023-03-24T22:22:22.527 INFO:teuthology.orchestra.run.smithi018.stdout:(128/142): NetworkManager-initscripts-updown-1. 5.6 MB/s | 142 kB 00:00 2023-03-24T22:22:22.583 INFO:teuthology.orchestra.run.smithi018.stdout:(129/142): elfutils-debuginfod-client-0.188-3.e 1.3 MB/s | 75 kB 00:00 2023-03-24T22:22:22.619 INFO:teuthology.orchestra.run.smithi018.stdout:(130/142): elfutils-libelf-0.188-3.el8.x86_64.r 6.3 MB/s | 231 kB 00:00 2023-03-24T22:22:22.653 INFO:teuthology.orchestra.run.smithi018.stdout:(131/142): elfutils-libs-0.188-3.el8.x86_64.rpm 8.8 MB/s | 299 kB 00:00 2023-03-24T22:22:22.678 INFO:teuthology.orchestra.run.smithi018.stdout:(132/142): libblkid-2.32.1-41.el8.x86_64.rpm 8.7 MB/s | 220 kB 00:00 2023-03-24T22:22:22.737 INFO:teuthology.orchestra.run.smithi018.stdout:(133/142): librados2-18.0.0-3124.gc19e8f2a.el8. 7.4 MB/s | 3.7 MB 00:00 2023-03-24T22:22:22.762 INFO:teuthology.orchestra.run.smithi018.stdout:(134/142): libfdisk-2.32.1-41.el8.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2023-03-24T22:22:22.787 INFO:teuthology.orchestra.run.smithi018.stdout:(135/142): libmount-2.32.1-41.el8.x86_64.rpm 4.6 MB/s | 236 kB 00:00 2023-03-24T22:22:22.813 INFO:teuthology.orchestra.run.smithi018.stdout:(136/142): libsmartcols-2.32.1-41.el8.x86_64.rp 3.5 MB/s | 179 kB 00:00 2023-03-24T22:22:22.830 INFO:teuthology.orchestra.run.smithi018.stdout:(137/142): libuuid-2.32.1-41.el8.x86_64.rpm 2.3 MB/s | 98 kB 00:00 2023-03-24T22:22:22.889 INFO:teuthology.orchestra.run.smithi018.stdout:(138/142): librbd1-18.0.0-3124.gc19e8f2a.el8.x8 5.7 MB/s | 3.6 MB 00:00 2023-03-24T22:22:22.930 INFO:teuthology.orchestra.run.smithi018.stdout:(139/142): openssl-1.1.1k-9.el8.x86_64.rpm 6.2 MB/s | 737 kB 00:00 2023-03-24T22:22:22.994 INFO:teuthology.orchestra.run.smithi018.stdout:(140/142): util-linux-user-2.32.1-41.el8.x86_64 1.6 MB/s | 102 kB 00:00 2023-03-24T22:22:23.031 INFO:teuthology.orchestra.run.smithi018.stdout:(141/142): openssl-libs-1.1.1k-9.el8.x86_64.rpm 7.3 MB/s | 1.5 MB 00:00 2023-03-24T22:22:23.032 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2023-03-24T22:22:23.036 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2023-03-24T22:22:23.081 INFO:teuthology.orchestra.run.smithi018.stdout:(142/142): util-linux-2.32.1-41.el8.x86_64.rpm 14 MB/s | 2.6 MB 00:00 2023-03-24T22:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2023-03-24T22:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:Total 16 MB/s | 204 MB 00:12 2023-03-24T22:22:24.021 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2023-03-24T22:22:24.310 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2023-03-24T22:22:24.310 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2023-03-24T22:22:25.863 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2023-03-24T22:22:25.867 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2023-03-24T22:22:29.101 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2023-03-24T22:22:29.266 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/1 2023-03-24T22:22:29.516 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-24T22:22:29.732 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-24T22:22:29.988 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/157 2023-03-24T22:22:30.134 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-24T22:22:30.207 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-24T22:22:30.392 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-24T22:22:30.447 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-24T22:22:30.682 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/157 2023-03-24T22:22:30.858 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-24T22:22:30.922 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-24T22:22:31.188 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/157 2023-03-24T22:22:31.391 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-24T22:22:31.698 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2023-03-24T22:22:31.818 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-24T22:22:31.863 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/1 2023-03-24T22:22:32.066 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T22:22:32.087 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-24T22:22:32.314 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-24T22:22:32.447 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T22:22:32.496 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/157 2023-03-24T22:22:32.633 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T22:22:32.675 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-24T22:22:32.746 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T22:22:32.760 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-24T22:22:32.933 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T22:22:32.941 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-24T22:22:33.052 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-24T22:22:33.068 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T22:22:33.250 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/157 2023-03-24T22:22:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 12/157 2023-03-24T22:22:33.441 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-24T22:22:33.494 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-24T22:22:33.546 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpq-13.5-1.el8.x86_64 13/157 2023-03-24T22:22:33.690 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/157 2023-03-24T22:22:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libelf-0.188-3.el8.x86_64 14/157 2023-03-24T22:22:33.874 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-24T22:22:34.119 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/157 2023-03-24T22:22:34.296 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-24T22:22:34.507 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T22:22:34.533 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/157 2023-03-24T22:22:34.787 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/157 2023-03-24T22:22:34.922 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T22:22:34.949 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 18/157 2023-03-24T22:22:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 18/157 2023-03-24T22:22:35.332 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T22:22:35.463 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T22:22:35.666 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T22:22:35.669 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 19/157 2023-03-24T22:22:35.767 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T22:22:35.999 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 12/157 2023-03-24T22:22:36.004 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 19/157 2023-03-24T22:22:36.194 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libpq-13.5-1.el8.x86_64 13/157 2023-03-24T22:22:36.272 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/157 2023-03-24T22:22:36.433 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-19.el8.x86_64 21/157 2023-03-24T22:22:36.453 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : elfutils-libelf-0.188-3.el8.x86_64 14/157 2023-03-24T22:22:36.599 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-24T22:22:36.730 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/157 2023-03-24T22:22:36.751 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-24T22:22:36.966 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-24T22:22:37.142 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/157 2023-03-24T22:22:37.379 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/157 2023-03-24T22:22:37.540 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 18/157 2023-03-24T22:22:37.553 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-24T22:22:37.658 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 18/157 2023-03-24T22:22:37.790 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/157 2023-03-24T22:22:37.815 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 19/157 2023-03-24T22:22:37.980 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 19/157 2023-03-24T22:22:38.213 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/157 2023-03-24T22:22:38.395 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/157 2023-03-24T22:22:38.405 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : zlib-devel-1.2.11-19.el8.x86_64 21/157 2023-03-24T22:22:38.565 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-24T22:22:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-24T22:22:38.697 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-24T22:22:38.873 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-24T22:22:39.258 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-24T22:22:39.505 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-24T22:22:39.674 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/157 2023-03-24T22:22:39.698 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/157 2023-03-24T22:22:39.951 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/157 2023-03-24T22:22:40.177 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/157 2023-03-24T22:22:40.346 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/157 2023-03-24T22:22:40.423 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/157 2023-03-24T22:22:40.515 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-24T22:22:40.655 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8. 30/157 2023-03-24T22:22:40.868 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el 31/157 2023-03-24T22:22:41.133 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_6 32/157 2023-03-24T22:22:41.461 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-24T22:22:41.846 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/157 2023-03-24T22:22:42.119 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/157 2023-03-24T22:22:42.132 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/157 2023-03-24T22:22:42.360 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/157 2023-03-24T22:22:42.579 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/157 2023-03-24T22:22:42.585 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/157 2023-03-24T22:22:42.779 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8. 30/157 2023-03-24T22:22:42.918 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/157 2023-03-24T22:22:42.944 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el 31/157 2023-03-24T22:22:43.156 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-24T22:22:43.205 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_6 32/157 2023-03-24T22:22:44.233 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/157 2023-03-24T22:22:44.661 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/157 2023-03-24T22:22:44.931 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-24T22:22:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/157 2023-03-24T22:22:45.239 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-24T22:22:45.293 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/157 2023-03-24T22:22:45.507 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/157 2023-03-24T22:22:45.792 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/157 2023-03-24T22:22:46.082 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libs-0.188-3.el8.x86_64 40/157 2023-03-24T22:22:46.299 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-debuginfod-client-0.188-3.el8.x86_64 41/157 2023-03-24T22:22:46.512 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 42/157 2023-03-24T22:22:46.737 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_6 43/157 2023-03-24T22:22:46.956 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-24T22:22:47.031 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-24T22:22:47.115 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-24T22:22:47.230 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libmount-2.32.1-41.el8.x86_64 45/157 2023-03-24T22:22:47.298 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 45/157 2023-03-24T22:22:47.478 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/157 2023-03-24T22:22:47.492 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.32.1-41.el8.x86_64 46/157 2023-03-24T22:22:47.684 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/157 2023-03-24T22:22:47.685 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/157 2023-03-24T22:22:47.889 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-24T22:22:47.927 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/157 2023-03-24T22:22:48.125 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : elfutils-libs-0.188-3.el8.x86_64 40/157 2023-03-24T22:22:48.343 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : elfutils-debuginfod-client-0.188-3.el8.x86_64 41/157 2023-03-24T22:22:48.372 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-24T22:22:48.537 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 42/157 2023-03-24T22:22:48.765 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_6 43/157 2023-03-24T22:22:48.772 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-24T22:22:48.787 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-24T22:22:48.955 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-24T22:22:49.016 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 50/157 2023-03-24T22:22:49.036 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-24T22:22:49.246 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libmount-2.32.1-41.el8.x86_64 45/157 2023-03-24T22:22:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 51/157 2023-03-24T22:22:49.329 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 45/157 2023-03-24T22:22:49.555 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libuuid-devel-2.32.1-41.el8.x86_64 46/157 2023-03-24T22:22:49.570 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 52/157 2023-03-24T22:22:49.734 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/157 2023-03-24T22:22:49.759 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 53/157 2023-03-24T22:22:49.940 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 54/157 2023-03-24T22:22:49.946 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-24T22:22:50.253 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 55/157 2023-03-24T22:22:50.470 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-24T22:22:50.507 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 56/157 2023-03-24T22:22:50.781 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 57/157 2023-03-24T22:22:50.813 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-24T22:22:50.829 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-24T22:22:51.025 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 50/157 2023-03-24T22:22:51.058 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 58/157 2023-03-24T22:22:51.303 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 59/157 2023-03-24T22:22:51.315 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 51/157 2023-03-24T22:22:51.525 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 60/157 2023-03-24T22:22:51.570 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 52/157 2023-03-24T22:22:51.759 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 53/157 2023-03-24T22:22:51.786 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 61/157 2023-03-24T22:22:51.974 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 54/157 2023-03-24T22:22:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-2.6-1.el8.noarch 62/157 2023-03-24T22:22:52.214 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 55/157 2023-03-24T22:22:52.271 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 63/157 2023-03-24T22:22:52.467 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 56/157 2023-03-24T22:22:52.504 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 64/157 2023-03-24T22:22:52.733 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 57/157 2023-03-24T22:22:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 65/157 2023-03-24T22:22:52.966 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 58/157 2023-03-24T22:22:53.141 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 66/157 2023-03-24T22:22:53.194 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 59/157 2023-03-24T22:22:53.401 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 60/157 2023-03-24T22:22:53.540 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 67/157 2023-03-24T22:22:53.635 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 61/157 2023-03-24T22:22:53.846 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-portend-2.6-1.el8.noarch 62/157 2023-03-24T22:22:53.863 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 68/157 2023-03-24T22:22:54.088 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 63/157 2023-03-24T22:22:54.104 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/157 2023-03-24T22:22:54.358 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 64/157 2023-03-24T22:22:54.631 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 65/157 2023-03-24T22:22:54.916 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 66/157 2023-03-24T22:22:54.953 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 70/157 2023-03-24T22:22:55.171 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 71/157 2023-03-24T22:22:55.247 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 71/157 2023-03-24T22:22:55.298 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 67/157 2023-03-24T22:22:55.524 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 72/157 2023-03-24T22:22:55.647 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 68/157 2023-03-24T22:22:55.789 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 73/157 2023-03-24T22:22:55.874 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/157 2023-03-24T22:22:56.269 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 74/157 2023-03-24T22:22:56.557 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 75/157 2023-03-24T22:22:56.772 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 70/157 2023-03-24T22:22:56.822 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 76/157 2023-03-24T22:22:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 71/157 2023-03-24T22:22:57.025 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 71/157 2023-03-24T22:22:57.099 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 77/157 2023-03-24T22:22:57.293 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 72/157 2023-03-24T22:22:57.400 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 78/157 2023-03-24T22:22:57.576 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 73/157 2023-03-24T22:22:57.999 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 74/157 2023-03-24T22:22:58.283 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 75/157 2023-03-24T22:22:58.507 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/157 2023-03-24T22:22:58.574 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 76/157 2023-03-24T22:22:58.873 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 77/157 2023-03-24T22:22:58.987 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 80/157 2023-03-24T22:22:59.160 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 78/157 2023-03-24T22:22:59.287 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 81/157 2023-03-24T22:22:59.587 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 82/157 2023-03-24T22:22:59.968 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 83/157 2023-03-24T22:23:00.254 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/157 2023-03-24T22:23:00.296 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/157 2023-03-24T22:23:00.539 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 85/157 2023-03-24T22:23:00.728 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 80/157 2023-03-24T22:23:00.793 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 86/157 2023-03-24T22:23:01.011 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 81/157 2023-03-24T22:23:01.125 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 87/157 2023-03-24T22:23:01.340 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 82/157 2023-03-24T22:23:01.555 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 88/157 2023-03-24T22:23:01.702 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 83/157 2023-03-24T22:23:01.761 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/157 2023-03-24T22:23:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/157 2023-03-24T22:23:02.086 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/157 2023-03-24T22:23:02.265 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 85/157 2023-03-24T22:23:02.357 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libselinux-devel-2.9-6.el8.x86_64 91/157 2023-03-24T22:23:02.498 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 86/157 2023-03-24T22:23:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/157 2023-03-24T22:23:02.756 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/157 2023-03-24T22:23:02.832 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 87/157 2023-03-24T22:23:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/157 2023-03-24T22:23:03.038 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 88/157 2023-03-24T22:23:03.221 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.7-16.el8.x86_64 95/157 2023-03-24T22:23:03.254 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/157 2023-03-24T22:23:03.432 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/157 2023-03-24T22:23:03.562 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/157 2023-03-24T22:23:03.628 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/157 2023-03-24T22:23:03.720 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/157 2023-03-24T22:23:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libselinux-devel-2.9-6.el8.x86_64 91/157 2023-03-24T22:23:03.895 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 98/157 2023-03-24T22:23:03.991 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/157 2023-03-24T22:23:04.189 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/157 2023-03-24T22:23:04.421 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/157 2023-03-24T22:23:04.629 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : fuse-2.9.7-16.el8.x86_64 95/157 2023-03-24T22:23:04.880 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/157 2023-03-24T22:23:05.052 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/157 2023-03-24T22:23:05.148 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/157 2023-03-24T22:23:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 98/157 2023-03-24T22:23:07.488 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T22:23:08.778 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T22:23:09.370 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T22:23:09.705 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T22:23:09.743 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 100/157 2023-03-24T22:23:09.860 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 100/157 2023-03-24T22:23:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T22:23:10.052 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 101/157 2023-03-24T22:23:10.603 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 99/157 2023-03-24T22:23:10.931 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 100/157 2023-03-24T22:23:11.044 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 100/157 2023-03-24T22:23:11.234 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 101/157 2023-03-24T22:23:29.578 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 101/157 2023-03-24T22:23:29.578 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2023-03-24T22:23:29.579 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2023-03-24T22:23:29.579 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /dev 2023-03-24T22:23:29.579 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /run 2023-03-24T22:23:29.579 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2023-03-24T22:23:29.579 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2023-03-24T22:23:29.579 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2023-03-24T22:23:29.579 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2023-03-24T22:23:29.580 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /dev 2023-03-24T22:23:29.580 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:23:29.974 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 102/157 2023-03-24T22:23:31.171 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 101/157 2023-03-24T22:23:31.171 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /sys 2023-03-24T22:23:31.171 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /proc 2023-03-24T22:23:31.172 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /dev 2023-03-24T22:23:31.172 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /run 2023-03-24T22:23:31.172 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /mnt 2023-03-24T22:23:31.172 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /var/tmp 2023-03-24T22:23:31.172 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /home 2023-03-24T22:23:31.172 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /tmp 2023-03-24T22:23:31.173 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /dev 2023-03-24T22:23:31.173 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:23:31.543 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 102/157 2023-03-24T22:23:31.648 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 102/157 2023-03-24T22:23:31.798 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 103/157 2023-03-24T22:23:32.016 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 103/157 2023-03-24T22:23:32.273 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 104/157 2023-03-24T22:23:32.726 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 104/157 2023-03-24T22:23:32.932 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 105/157 2023-03-24T22:23:32.956 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 105/157 2023-03-24T22:23:33.238 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 106/157 2023-03-24T22:23:33.311 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 102/157 2023-03-24T22:23:33.501 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 107/157 2023-03-24T22:23:33.561 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 103/157 2023-03-24T22:23:33.664 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-24T22:23:33.786 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-24T22:23:33.792 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 103/157 2023-03-24T22:23:33.981 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 104/157 2023-03-24T22:23:34.078 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : jq-1.6-6.el8.x86_64 109/157 2023-03-24T22:23:34.369 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.1-7.el8.noarch 110/157 2023-03-24T22:23:34.459 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 104/157 2023-03-24T22:23:34.664 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 105/157 2023-03-24T22:23:34.689 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 105/157 2023-03-24T22:23:34.930 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 106/157 2023-03-24T22:23:35.193 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 107/157 2023-03-24T22:23:35.388 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-24T22:23:35.467 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-24T22:23:35.740 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : jq-1.6-6.el8.x86_64 109/157 2023-03-24T22:23:35.938 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 111/157 2023-03-24T22:23:36.033 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : automake-1.16.1-7.el8.noarch 110/157 2023-03-24T22:23:36.141 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 111/157 2023-03-24T22:23:36.329 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.e 112/157 2023-03-24T22:23:36.590 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a. 113/157 2023-03-24T22:23:36.806 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 114/157 2023-03-24T22:23:37.017 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 114/157 2023-03-24T22:23:37.280 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 115/157 2023-03-24T22:23:37.731 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 111/157 2023-03-24T22:23:37.921 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 111/157 2023-03-24T22:23:38.078 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.e 112/157 2023-03-24T22:23:38.248 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 115/157 2023-03-24T22:23:38.320 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a. 113/157 2023-03-24T22:23:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 116/157 2023-03-24T22:23:38.546 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 114/157 2023-03-24T22:23:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 116/157 2023-03-24T22:23:38.768 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 114/157 2023-03-24T22:23:39.051 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 115/157 2023-03-24T22:23:39.098 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el 117/157 2023-03-24T22:23:39.350 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 118/157 2023-03-24T22:23:39.385 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 118/157 2023-03-24T22:23:39.614 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 119/157 2023-03-24T22:23:39.625 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 119/157 2023-03-24T22:23:39.832 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 120/157 2023-03-24T22:23:40.072 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-25.el8.x86_64 121/157 2023-03-24T22:23:40.085 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 115/157 2023-03-24T22:23:40.296 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 116/157 2023-03-24T22:23:40.393 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 116/157 2023-03-24T22:23:40.836 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el 117/157 2023-03-24T22:23:41.073 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 118/157 2023-03-24T22:23:41.110 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 118/157 2023-03-24T22:23:41.288 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 119/157 2023-03-24T22:23:41.299 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 119/157 2023-03-24T22:23:41.528 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 120/157 2023-03-24T22:23:41.712 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libtool-2.4.6-25.el8.x86_64 121/157 2023-03-24T22:23:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 121/157 2023-03-24T22:23:45.508 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 122/157 2023-03-24T22:23:45.766 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 123/157 2023-03-24T22:23:45.800 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 123/157 2023-03-24T22:23:45.849 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 121/157 2023-03-24T22:23:46.046 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 124/157 2023-03-24T22:23:46.088 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 124/157 2023-03-24T22:23:46.346 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 125/157 2023-03-24T22:23:46.693 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 125/157 2023-03-24T22:23:47.004 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 126/157 2023-03-24T22:23:47.216 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 126/157 2023-03-24T22:23:47.362 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 122/157 2023-03-24T22:23:47.588 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 123/157 2023-03-24T22:23:47.612 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 127/157 2023-03-24T22:23:47.623 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 123/157 2023-03-24T22:23:47.879 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 124/157 2023-03-24T22:23:47.923 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 124/157 2023-03-24T22:23:48.137 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 125/157 2023-03-24T22:23:48.504 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 125/157 2023-03-24T22:23:48.510 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 128/157 2023-03-24T22:23:48.753 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 126/157 2023-03-24T22:23:48.843 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 129/157 2023-03-24T22:23:48.976 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 126/157 2023-03-24T22:23:49.056 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-user-2.32.1-41.el8.x86_64 130/157 2023-03-24T22:23:49.258 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 131/157 2023-03-24T22:23:49.365 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 127/157 2023-03-24T22:23:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_ 132/157 2023-03-24T22:23:49.705 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.188-3.el8.x86_64 133/157 2023-03-24T22:23:50.339 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 128/157 2023-03-24T22:23:50.676 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 129/157 2023-03-24T22:23:50.940 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : util-linux-user-2.32.1-41.el8.x86_64 130/157 2023-03-24T22:23:51.124 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 131/157 2023-03-24T22:23:51.349 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_ 132/157 2023-03-24T22:23:51.546 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : elfutils-libelf-devel-0.188-3.el8.x86_64 133/157 2023-03-24T22:23:51.788 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T22:23:52.137 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T22:23:52.155 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T22:23:52.487 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 135/157 2023-03-24T22:23:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 136/157 2023-03-24T22:23:53.024 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 137/157 2023-03-24T22:23:53.422 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:1.1.1k-9.el8.x86_64 138/157 2023-03-24T22:23:53.578 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-24T22:23:53.592 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-24T22:23:53.773 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T22:23:53.807 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 140/157 2023-03-24T22:23:54.036 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.1-9.el8.x86_64 141/157 2023-03-24T22:23:54.061 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T22:23:54.085 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-24T22:23:54.116 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 141/157 2023-03-24T22:23:54.344 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.0.4-10.el8.x86_64 142/157 2023-03-24T22:23:54.359 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 142/157 2023-03-24T22:23:54.369 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 135/157 2023-03-24T22:23:54.613 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:1.1.1k-7.el8.x86_64 143/157 2023-03-24T22:23:54.619 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 136/157 2023-03-24T22:23:54.804 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-debuginfod-client-0.187-4.el8.x86_64 144/157 2023-03-24T22:23:54.957 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 137/157 2023-03-24T22:23:55.032 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libs-0.187-4.el8.x86_64 145/157 2023-03-24T22:23:55.033 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-24T22:23:55.288 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-24T22:23:55.293 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-24T22:23:55.296 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : openssl-1:1.1.1k-9.el8.x86_64 138/157 2023-03-24T22:23:55.406 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-24T22:23:55.406 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-24T22:23:55.469 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-24T22:23:55.482 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-24T22:23:55.688 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-24T22:23:55.731 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 140/157 2023-03-24T22:23:55.836 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-user-2.32.1-38.el8.x86_64 149/157 2023-03-24T22:23:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flex-2.6.1-9.el8.x86_64 141/157 2023-03-24T22:23:56.031 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 141/157 2023-03-24T22:23:56.129 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-2.32.1-38.el8.x86_64 150/157 2023-03-24T22:23:56.130 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-24T22:23:56.235 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : bison-3.0.4-10.el8.x86_64 142/157 2023-03-24T22:23:56.252 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 142/157 2023-03-24T22:23:56.439 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-24T22:23:56.440 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libmount-2.32.1-38.el8.x86_64 152/157 2023-03-24T22:23:56.504 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-1:1.1.1k-7.el8.x86_64 143/157 2023-03-24T22:23:56.662 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : elfutils-debuginfod-client-0.187-4.el8.x86_64 144/157 2023-03-24T22:23:56.688 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libmount-2.32.1-38.el8.x86_64 152/157 2023-03-24T22:23:56.688 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-24T22:23:56.896 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : elfutils-libs-0.187-4.el8.x86_64 145/157 2023-03-24T22:23:56.897 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-24T22:23:56.938 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-24T22:23:56.938 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-24T22:23:57.129 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-24T22:23:57.135 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-24T22:23:57.155 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-24T22:23:57.155 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-24T22:23:57.280 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-24T22:23:57.281 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-24T22:23:57.372 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-24T22:23:57.538 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-24T22:23:57.588 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libelf-0.187-4.el8.x86_64 156/157 2023-03-24T22:23:57.589 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-24T22:23:57.724 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : util-linux-user-2.32.1-38.el8.x86_64 149/157 2023-03-24T22:23:57.991 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : util-linux-2.32.1-38.el8.x86_64 150/157 2023-03-24T22:23:57.991 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-24T22:23:58.221 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-24T22:23:58.221 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libmount-2.32.1-38.el8.x86_64 152/157 2023-03-24T22:23:58.488 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libmount-2.32.1-38.el8.x86_64 152/157 2023-03-24T22:23:58.488 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-24T22:23:58.565 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-24T22:23:58.565 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 1/157 2023-03-24T22:23:58.565 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2/157 2023-03-24T22:23:58.566 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 3/157 2023-03-24T22:23:58.566 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 4/157 2023-03-24T22:23:58.566 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 5/157 2023-03-24T22:23:58.566 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 6/157 2023-03-24T22:23:58.566 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 7/157 2023-03-24T22:23:58.566 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 8/157 2023-03-24T22:23:58.566 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T22:23:58.567 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T22:23:58.567 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T22:23:58.567 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 12/157 2023-03-24T22:23:58.567 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 13/157 2023-03-24T22:23:58.567 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_ 14/157 2023-03-24T22:23:58.567 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 15/157 2023-03-24T22:23:58.567 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 16/157 2023-03-24T22:23:58.568 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_6 17/157 2023-03-24T22:23:58.568 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 18/157 2023-03-24T22:23:58.568 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 19/157 2023-03-24T22:23:58.568 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el 20/157 2023-03-24T22:23:58.568 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8. 21/157 2023-03-24T22:23:58.568 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_6 22/157 2023-03-24T22:23:58.568 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 23/157 2023-03-24T22:23:58.568 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 24/157 2023-03-24T22:23:58.569 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 25/157 2023-03-24T22:23:58.569 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 26/157 2023-03-24T22:23:58.569 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 27/157 2023-03-24T22:23:58.569 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 28/157 2023-03-24T22:23:58.569 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a. 29/157 2023-03-24T22:23:58.569 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 30/157 2023-03-24T22:23:58.577 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 31/157 2023-03-24T22:23:58.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 32/157 2023-03-24T22:23:58.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el 33/157 2023-03-24T22:23:58.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 34/157 2023-03-24T22:23:58.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.e 35/157 2023-03-24T22:23:58.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 36/157 2023-03-24T22:23:58.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 37/157 2023-03-24T22:23:58.578 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 38/157 2023-03-24T22:23:58.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/157 2023-03-24T22:23:58.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/157 2023-03-24T22:23:58.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/157 2023-03-24T22:23:58.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : jq-1.6-6.el8.x86_64 42/157 2023-03-24T22:23:58.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/157 2023-03-24T22:23:58.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/157 2023-03-24T22:23:58.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/157 2023-03-24T22:23:58.579 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/157 2023-03-24T22:23:58.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/157 2023-03-24T22:23:58.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/157 2023-03-24T22:23:58.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/157 2023-03-24T22:23:58.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/157 2023-03-24T22:23:58.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/157 2023-03-24T22:23:58.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/157 2023-03-24T22:23:58.580 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/157 2023-03-24T22:23:58.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/157 2023-03-24T22:23:58.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/157 2023-03-24T22:23:58.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/157 2023-03-24T22:23:58.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/157 2023-03-24T22:23:58.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/157 2023-03-24T22:23:58.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/157 2023-03-24T22:23:58.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/157 2023-03-24T22:23:58.581 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/157 2023-03-24T22:23:58.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/157 2023-03-24T22:23:58.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/157 2023-03-24T22:23:58.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/157 2023-03-24T22:23:58.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/157 2023-03-24T22:23:58.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/157 2023-03-24T22:23:58.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/157 2023-03-24T22:23:58.582 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/157 2023-03-24T22:23:58.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/157 2023-03-24T22:23:58.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/157 2023-03-24T22:23:58.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.188-3.el8.x86_64 71/157 2023-03-24T22:23:58.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 72/157 2023-03-24T22:23:58.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/157 2023-03-24T22:23:58.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/157 2023-03-24T22:23:58.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/157 2023-03-24T22:23:58.583 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/157 2023-03-24T22:23:58.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 77/157 2023-03-24T22:23:58.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 78/157 2023-03-24T22:23:58.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 79/157 2023-03-24T22:23:58.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libselinux-devel-2.9-6.el8.x86_64 80/157 2023-03-24T22:23:58.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/157 2023-03-24T22:23:58.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.32.1-41.el8.x86_64 82/157 2023-03-24T22:23:58.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/157 2023-03-24T22:23:58.584 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 84/157 2023-03-24T22:23:58.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 85/157 2023-03-24T22:23:58.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 86/157 2023-03-24T22:23:58.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 87/157 2023-03-24T22:23:58.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/157 2023-03-24T22:23:58.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 89/157 2023-03-24T22:23:58.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 90/157 2023-03-24T22:23:58.585 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 91/157 2023-03-24T22:23:58.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 92/157 2023-03-24T22:23:58.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 93/157 2023-03-24T22:23:58.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 94/157 2023-03-24T22:23:58.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-19.el8.x86_64 95/157 2023-03-24T22:23:58.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 96/157 2023-03-24T22:23:58.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 97/157 2023-03-24T22:23:58.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 98/157 2023-03-24T22:23:58.586 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 99/157 2023-03-24T22:23:58.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 100/157 2023-03-24T22:23:58.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 101/157 2023-03-24T22:23:58.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 102/157 2023-03-24T22:23:58.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 103/157 2023-03-24T22:23:58.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 104/157 2023-03-24T22:23:58.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 105/157 2023-03-24T22:23:58.587 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 106/157 2023-03-24T22:23:58.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 107/157 2023-03-24T22:23:58.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 108/157 2023-03-24T22:23:58.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 109/157 2023-03-24T22:23:58.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 110/157 2023-03-24T22:23:58.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 111/157 2023-03-24T22:23:58.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 112/157 2023-03-24T22:23:58.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 113/157 2023-03-24T22:23:58.588 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-2.6-1.el8.noarch 114/157 2023-03-24T22:23:58.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 115/157 2023-03-24T22:23:58.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 116/157 2023-03-24T22:23:58.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 117/157 2023-03-24T22:23:58.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 118/157 2023-03-24T22:23:58.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 119/157 2023-03-24T22:23:58.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 120/157 2023-03-24T22:23:58.589 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 121/157 2023-03-24T22:23:58.590 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 122/157 2023-03-24T22:23:58.590 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 123/157 2023-03-24T22:23:58.590 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 124/157 2023-03-24T22:23:58.590 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 125/157 2023-03-24T22:23:58.590 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 126/157 2023-03-24T22:23:58.590 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 127/157 2023-03-24T22:23:58.590 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 128/157 2023-03-24T22:23:58.590 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 129/157 2023-03-24T22:23:58.591 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 130/157 2023-03-24T22:23:58.591 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 131/157 2023-03-24T22:23:58.591 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-1.el 132/157 2023-03-24T22:23:58.591 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-initscripts-updown-1:1.39.12-1.el 133/157 2023-03-24T22:23:58.591 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el8.x86_64 134/157 2023-03-24T22:23:58.591 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.187-4.el8.x86_64 135/157 2023-03-24T22:23:58.591 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.188-3.el8.x86_64 136/157 2023-03-24T22:23:58.591 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.187-4.el8.x86_64 137/157 2023-03-24T22:23:58.592 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.188-3.el8.x86_64 138/157 2023-03-24T22:23:58.592 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.187-4.el8.x86_64 139/157 2023-03-24T22:23:58.592 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 140/157 2023-03-24T22:23:58.592 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.32.1-38.el8.x86_64 141/157 2023-03-24T22:23:58.592 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 142/157 2023-03-24T22:23:58.592 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.32.1-38.el8.x86_64 143/157 2023-03-24T22:23:58.592 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 144/157 2023-03-24T22:23:58.593 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.32.1-38.el8.x86_64 145/157 2023-03-24T22:23:58.593 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 146/157 2023-03-24T22:23:58.593 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.32.1-38.el8.x86_64 147/157 2023-03-24T22:23:58.593 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 148/157 2023-03-24T22:23:58.593 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.32.1-38.el8.x86_64 149/157 2023-03-24T22:23:58.593 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 150/157 2023-03-24T22:23:58.593 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:1.1.1k-7.el8.x86_64 151/157 2023-03-24T22:23:58.593 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 152/157 2023-03-24T22:23:58.594 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8.x86_64 153/157 2023-03-24T22:23:58.594 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/157 2023-03-24T22:23:58.594 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.32.1-38.el8.x86_64 155/157 2023-03-24T22:23:58.594 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/157 2023-03-24T22:23:58.696 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-24T22:23:58.696 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-24T22:23:58.938 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-24T22:23:58.938 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-24T22:23:59.146 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-24T22:23:59.388 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : elfutils-libelf-0.187-4.el8.x86_64 156/157 2023-03-24T22:23:59.388 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-24T22:24:00.378 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-24T22:24:00.378 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 1/157 2023-03-24T22:24:00.378 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2/157 2023-03-24T22:24:00.378 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 3/157 2023-03-24T22:24:00.379 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 4/157 2023-03-24T22:24:00.379 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3124.gc19e8 5/157 2023-03-24T22:24:00.379 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 6/157 2023-03-24T22:24:00.379 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 7/157 2023-03-24T22:24:00.379 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 8/157 2023-03-24T22:24:00.379 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 9/157 2023-03-24T22:24:00.379 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 10/157 2023-03-24T22:24:00.380 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 11/157 2023-03-24T22:24:00.380 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 12/157 2023-03-24T22:24:00.380 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 13/157 2023-03-24T22:24:00.380 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_ 14/157 2023-03-24T22:24:00.380 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 15/157 2023-03-24T22:24:00.380 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 16/157 2023-03-24T22:24:00.380 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_6 17/157 2023-03-24T22:24:00.380 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86 18/157 2023-03-24T22:24:00.381 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 19/157 2023-03-24T22:24:00.381 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el 20/157 2023-03-24T22:24:00.381 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8. 21/157 2023-03-24T22:24:00.381 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_6 22/157 2023-03-24T22:24:00.382 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 23/157 2023-03-24T22:24:00.382 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 24/157 2023-03-24T22:24:00.382 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 25/157 2023-03-24T22:24:00.383 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 26/157 2023-03-24T22:24:00.383 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 27/157 2023-03-24T22:24:00.383 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 28/157 2023-03-24T22:24:00.383 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a. 29/157 2023-03-24T22:24:00.383 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noa 30/157 2023-03-24T22:24:00.383 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.n 31/157 2023-03-24T22:24:00.383 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19 32/157 2023-03-24T22:24:00.391 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el 33/157 2023-03-24T22:24:00.391 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 34/157 2023-03-24T22:24:00.392 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.e 35/157 2023-03-24T22:24:00.392 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 36/157 2023-03-24T22:24:00.392 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 37/157 2023-03-24T22:24:00.392 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 38/157 2023-03-24T22:24:00.392 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/157 2023-03-24T22:24:00.392 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/157 2023-03-24T22:24:00.392 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/157 2023-03-24T22:24:00.393 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : jq-1.6-6.el8.x86_64 42/157 2023-03-24T22:24:00.393 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/157 2023-03-24T22:24:00.393 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/157 2023-03-24T22:24:00.393 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/157 2023-03-24T22:24:00.393 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/157 2023-03-24T22:24:00.393 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/157 2023-03-24T22:24:00.393 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/157 2023-03-24T22:24:00.393 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/157 2023-03-24T22:24:00.394 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/157 2023-03-24T22:24:00.394 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/157 2023-03-24T22:24:00.394 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/157 2023-03-24T22:24:00.394 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/157 2023-03-24T22:24:00.394 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/157 2023-03-24T22:24:00.394 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/157 2023-03-24T22:24:00.394 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/157 2023-03-24T22:24:00.394 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/157 2023-03-24T22:24:00.395 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/157 2023-03-24T22:24:00.395 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/157 2023-03-24T22:24:00.395 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/157 2023-03-24T22:24:00.395 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/157 2023-03-24T22:24:00.395 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/157 2023-03-24T22:24:00.395 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/157 2023-03-24T22:24:00.395 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/157 2023-03-24T22:24:00.396 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/157 2023-03-24T22:24:00.396 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/157 2023-03-24T22:24:00.396 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/157 2023-03-24T22:24:00.396 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/157 2023-03-24T22:24:00.396 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/157 2023-03-24T22:24:00.396 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/157 2023-03-24T22:24:00.396 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-devel-0.188-3.el8.x86_64 71/157 2023-03-24T22:24:00.396 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 72/157 2023-03-24T22:24:00.397 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/157 2023-03-24T22:24:00.397 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/157 2023-03-24T22:24:00.397 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/157 2023-03-24T22:24:00.397 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/157 2023-03-24T22:24:00.397 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 77/157 2023-03-24T22:24:00.397 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 78/157 2023-03-24T22:24:00.397 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 79/157 2023-03-24T22:24:00.398 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libselinux-devel-2.9-6.el8.x86_64 80/157 2023-03-24T22:24:00.398 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/157 2023-03-24T22:24:00.398 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-devel-2.32.1-41.el8.x86_64 82/157 2023-03-24T22:24:00.398 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/157 2023-03-24T22:24:00.398 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 84/157 2023-03-24T22:24:00.398 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 85/157 2023-03-24T22:24:00.398 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 86/157 2023-03-24T22:24:00.398 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 87/157 2023-03-24T22:24:00.399 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/157 2023-03-24T22:24:00.399 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 89/157 2023-03-24T22:24:00.399 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 90/157 2023-03-24T22:24:00.399 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 91/157 2023-03-24T22:24:00.399 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 92/157 2023-03-24T22:24:00.399 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 93/157 2023-03-24T22:24:00.399 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 94/157 2023-03-24T22:24:00.399 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : zlib-devel-1.2.11-19.el8.x86_64 95/157 2023-03-24T22:24:00.400 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 96/157 2023-03-24T22:24:00.400 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 97/157 2023-03-24T22:24:00.400 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 98/157 2023-03-24T22:24:00.400 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 99/157 2023-03-24T22:24:00.400 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 100/157 2023-03-24T22:24:00.400 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 101/157 2023-03-24T22:24:00.400 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 102/157 2023-03-24T22:24:00.400 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 103/157 2023-03-24T22:24:00.401 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 104/157 2023-03-24T22:24:00.401 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 105/157 2023-03-24T22:24:00.401 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 106/157 2023-03-24T22:24:00.401 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 107/157 2023-03-24T22:24:00.401 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 108/157 2023-03-24T22:24:00.401 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 109/157 2023-03-24T22:24:00.401 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 110/157 2023-03-24T22:24:00.402 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 111/157 2023-03-24T22:24:00.402 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 112/157 2023-03-24T22:24:00.402 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 113/157 2023-03-24T22:24:00.402 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-portend-2.6-1.el8.noarch 114/157 2023-03-24T22:24:00.402 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 115/157 2023-03-24T22:24:00.402 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 116/157 2023-03-24T22:24:00.402 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 117/157 2023-03-24T22:24:00.402 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 118/157 2023-03-24T22:24:00.403 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 119/157 2023-03-24T22:24:00.403 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 120/157 2023-03-24T22:24:00.403 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 121/157 2023-03-24T22:24:00.403 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 122/157 2023-03-24T22:24:00.403 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 123/157 2023-03-24T22:24:00.403 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 124/157 2023-03-24T22:24:00.403 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 125/157 2023-03-24T22:24:00.403 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 126/157 2023-03-24T22:24:00.404 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 127/157 2023-03-24T22:24:00.404 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 128/157 2023-03-24T22:24:00.404 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 129/157 2023-03-24T22:24:00.404 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 130/157 2023-03-24T22:24:00.404 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 131/157 2023-03-24T22:24:00.404 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-1.el 132/157 2023-03-24T22:24:00.404 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-initscripts-updown-1:1.39.12-1.el 133/157 2023-03-24T22:24:00.404 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el8.x86_64 134/157 2023-03-24T22:24:00.405 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-debuginfod-client-0.187-4.el8.x86_64 135/157 2023-03-24T22:24:00.405 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-0.188-3.el8.x86_64 136/157 2023-03-24T22:24:00.405 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-0.187-4.el8.x86_64 137/157 2023-03-24T22:24:00.405 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libs-0.188-3.el8.x86_64 138/157 2023-03-24T22:24:00.405 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libs-0.187-4.el8.x86_64 139/157 2023-03-24T22:24:00.405 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 140/157 2023-03-24T22:24:00.405 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libblkid-2.32.1-38.el8.x86_64 141/157 2023-03-24T22:24:00.406 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 142/157 2023-03-24T22:24:00.406 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libfdisk-2.32.1-38.el8.x86_64 143/157 2023-03-24T22:24:00.406 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 144/157 2023-03-24T22:24:00.406 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libmount-2.32.1-38.el8.x86_64 145/157 2023-03-24T22:24:00.406 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 146/157 2023-03-24T22:24:00.406 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libsmartcols-2.32.1-38.el8.x86_64 147/157 2023-03-24T22:24:00.406 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 148/157 2023-03-24T22:24:00.406 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-2.32.1-38.el8.x86_64 149/157 2023-03-24T22:24:00.407 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 150/157 2023-03-24T22:24:00.407 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:1.1.1k-7.el8.x86_64 151/157 2023-03-24T22:24:00.407 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 152/157 2023-03-24T22:24:00.407 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8.x86_64 153/157 2023-03-24T22:24:00.407 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/157 2023-03-24T22:24:00.407 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-2.32.1-38.el8.x86_64 155/157 2023-03-24T22:24:00.407 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/157 2023-03-24T22:24:00.963 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.32.1-38.el8.x86_64 157/157 2023-03-24T22:24:00.964 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:00.964 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2023-03-24T22:24:00.964 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-initscripts-updown-1:1.40.16-1.el8.noarch 2023-03-24T22:24:00.964 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client-0.188-3.el8.x86_64 2023-03-24T22:24:00.964 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-0.188-3.el8.x86_64 2023-03-24T22:24:00.964 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs-0.188-3.el8.x86_64 2023-03-24T22:24:00.964 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid-2.32.1-41.el8.x86_64 2023-03-24T22:24:00.965 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk-2.32.1-41.el8.x86_64 2023-03-24T22:24:00.965 INFO:teuthology.orchestra.run.smithi100.stdout: libmount-2.32.1-41.el8.x86_64 2023-03-24T22:24:00.965 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.965 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.965 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols-2.32.1-41.el8.x86_64 2023-03-24T22:24:00.965 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-2.32.1-41.el8.x86_64 2023-03-24T22:24:00.966 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:1.1.1k-9.el8.x86_64 2023-03-24T22:24:00.966 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:1.1.1k-9.el8.x86_64 2023-03-24T22:24:00.966 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-2.32.1-41.el8.x86_64 2023-03-24T22:24:00.966 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user-2.32.1-41.el8.x86_64 2023-03-24T22:24:00.966 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2023-03-24T22:24:00.966 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.1-7.el8.noarch 2023-03-24T22:24:00.966 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.0.4-10.el8.x86_64 2023-03-24T22:24:00.966 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:00.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.967 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:00.968 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:00.968 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:00.968 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:00.968 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:00.968 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.969 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.969 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:00.969 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.969 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.970 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.970 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:00.970 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:00.971 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.971 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:00.971 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.188-3.el8.x86_64 2023-03-24T22:24:00.971 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.1-9.el8.x86_64 2023-03-24T22:24:00.972 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.7-16.el8.x86_64 2023-03-24T22:24:00.972 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-24T22:24:00.972 INFO:teuthology.orchestra.run.smithi100.stdout: jq-1.6-6.el8.x86_64 2023-03-24T22:24:00.972 INFO:teuthology.orchestra.run.smithi100.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-03-24T22:24:00.973 INFO:teuthology.orchestra.run.smithi100.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-03-24T22:24:00.973 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-03-24T22:24:00.973 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.974 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.974 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.974 INFO:teuthology.orchestra.run.smithi100.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-03-24T22:24:00.975 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-03-24T22:24:00.975 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-24T22:24:00.975 INFO:teuthology.orchestra.run.smithi100.stdout: libpq-13.5-1.el8.x86_64 2023-03-24T22:24:00.976 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-03-24T22:24:00.976 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-24T22:24:00.976 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.976 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.976 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-24T22:24:00.976 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.977 INFO:teuthology.orchestra.run.smithi100.stdout: libselinux-devel-2.9-6.el8.x86_64 2023-03-24T22:24:00.977 INFO:teuthology.orchestra.run.smithi100.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-03-24T22:24:00.977 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-25.el8.x86_64 2023-03-24T22:24:00.977 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-24T22:24:00.978 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.32.1-41.el8.x86_64 2023-03-24T22:24:00.978 INFO:teuthology.orchestra.run.smithi100.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-03-24T22:24:00.979 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-24T22:24:00.979 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-24T22:24:00.979 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.15-4.el8.x86_64 2023-03-24T22:24:00.979 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-03-24T22:24:00.980 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-03-24T22:24:00.980 INFO:teuthology.orchestra.run.smithi100.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-03-24T22:24:00.980 INFO:teuthology.orchestra.run.smithi100.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-03-24T22:24:00.981 INFO:teuthology.orchestra.run.smithi100.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-03-24T22:24:00.981 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T22:24:00.981 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T22:24:00.981 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T22:24:00.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-24T22:24:00.982 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-24T22:24:00.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-24T22:24:00.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-24T22:24:00.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-24T22:24:00.983 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-24T22:24:00.984 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-24T22:24:00.985 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-24T22:24:00.985 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-24T22:24:00.985 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-24T22:24:00.986 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-24T22:24:00.986 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-24T22:24:00.986 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-24T22:24:00.986 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-24T22:24:00.986 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-24T22:24:00.986 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-24T22:24:00.986 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-24T22:24:00.987 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-24T22:24:00.987 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-24T22:24:00.987 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.0.6-14.el8.noarch 2023-03-24T22:24:00.987 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-24T22:24:00.987 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-24T22:24:00.987 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-24T22:24:00.987 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-24T22:24:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-24T22:24:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-24T22:24:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-24T22:24:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-2.6-1.el8.noarch 2023-03-24T22:24:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-24T22:24:00.988 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-24T22:24:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-24T22:24:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-24T22:24:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-24T22:24:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-24T22:24:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.989 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-24T22:24:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-24T22:24:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-24T22:24:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-24T22:24:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-03-24T22:24:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-24T22:24:00.990 INFO:teuthology.orchestra.run.smithi100.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-24T22:24:00.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-24T22:24:00.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-24T22:24:00.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-24T22:24:00.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-24T22:24:00.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-24T22:24:00.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-24T22:24:00.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-24T22:24:00.991 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-24T22:24:00.992 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.992 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.992 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:00.992 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-24T22:24:00.992 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-24T22:24:00.992 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-43.el8.x86_64 2023-03-24T22:24:00.992 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-03-24T22:24:00.992 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-24T22:24:00.993 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-19.el8.x86_64 2023-03-24T22:24:00.993 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:00.994 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2023-03-24T22:24:01.143 DEBUG:teuthology.parallel:result is None 2023-03-24T22:24:02.528 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : util-linux-user-2.32.1-38.el8.x86_64 157/157 2023-03-24T22:24:02.528 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:02.528 INFO:teuthology.orchestra.run.smithi018.stdout:Upgraded: 2023-03-24T22:24:02.529 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-initscripts-updown-1:1.40.16-1.el8.noarch 2023-03-24T22:24:02.529 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-debuginfod-client-0.188-3.el8.x86_64 2023-03-24T22:24:02.529 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-0.188-3.el8.x86_64 2023-03-24T22:24:02.529 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libs-0.188-3.el8.x86_64 2023-03-24T22:24:02.529 INFO:teuthology.orchestra.run.smithi018.stdout: libblkid-2.32.1-41.el8.x86_64 2023-03-24T22:24:02.529 INFO:teuthology.orchestra.run.smithi018.stdout: libfdisk-2.32.1-41.el8.x86_64 2023-03-24T22:24:02.530 INFO:teuthology.orchestra.run.smithi018.stdout: libmount-2.32.1-41.el8.x86_64 2023-03-24T22:24:02.530 INFO:teuthology.orchestra.run.smithi018.stdout: librados2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.530 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.530 INFO:teuthology.orchestra.run.smithi018.stdout: libsmartcols-2.32.1-41.el8.x86_64 2023-03-24T22:24:02.530 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-2.32.1-41.el8.x86_64 2023-03-24T22:24:02.530 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-1:1.1.1k-9.el8.x86_64 2023-03-24T22:24:02.530 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs-1:1.1.1k-9.el8.x86_64 2023-03-24T22:24:02.530 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-2.32.1-41.el8.x86_64 2023-03-24T22:24:02.531 INFO:teuthology.orchestra.run.smithi018.stdout: util-linux-user-2.32.1-41.el8.x86_64 2023-03-24T22:24:02.531 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2023-03-24T22:24:02.531 INFO:teuthology.orchestra.run.smithi018.stdout: automake-1.16.1-7.el8.noarch 2023-03-24T22:24:02.531 INFO:teuthology.orchestra.run.smithi018.stdout: bison-3.0.4-10.el8.x86_64 2023-03-24T22:24:02.531 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.531 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.532 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.532 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.532 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:02.532 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.533 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.533 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.533 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:02.533 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:02.533 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:02.533 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:02.534 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:02.534 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.534 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.534 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:02.535 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.535 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.535 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.535 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:02.536 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:02.536 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.536 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-top-2:18.0.0-3124.gc19e8f2a.el8.noarch 2023-03-24T22:24:02.536 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel-0.188-3.el8.x86_64 2023-03-24T22:24:02.537 INFO:teuthology.orchestra.run.smithi018.stdout: flex-2.6.1-9.el8.x86_64 2023-03-24T22:24:02.537 INFO:teuthology.orchestra.run.smithi018.stdout: fuse-2.9.7-16.el8.x86_64 2023-03-24T22:24:02.537 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-24T22:24:02.537 INFO:teuthology.orchestra.run.smithi018.stdout: jq-1.6-6.el8.x86_64 2023-03-24T22:24:02.537 INFO:teuthology.orchestra.run.smithi018.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-03-24T22:24:02.538 INFO:teuthology.orchestra.run.smithi018.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-03-24T22:24:02.538 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-03-24T22:24:02.538 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.538 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.538 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.538 INFO:teuthology.orchestra.run.smithi018.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-03-24T22:24:02.538 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-03-24T22:24:02.538 INFO:teuthology.orchestra.run.smithi018.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-24T22:24:02.539 INFO:teuthology.orchestra.run.smithi018.stdout: libpq-13.5-1.el8.x86_64 2023-03-24T22:24:02.539 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-03-24T22:24:02.539 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-24T22:24:02.539 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.539 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.539 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-24T22:24:02.540 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.540 INFO:teuthology.orchestra.run.smithi018.stdout: libselinux-devel-2.9-6.el8.x86_64 2023-03-24T22:24:02.540 INFO:teuthology.orchestra.run.smithi018.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-03-24T22:24:02.540 INFO:teuthology.orchestra.run.smithi018.stdout: libtool-2.4.6-25.el8.x86_64 2023-03-24T22:24:02.540 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-24T22:24:02.541 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel-2.32.1-41.el8.x86_64 2023-03-24T22:24:02.541 INFO:teuthology.orchestra.run.smithi018.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-03-24T22:24:02.541 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-24T22:24:02.541 INFO:teuthology.orchestra.run.smithi018.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-24T22:24:02.541 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-0.3.15-4.el8.x86_64 2023-03-24T22:24:02.541 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-03-24T22:24:02.541 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-03-24T22:24:02.542 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-03-24T22:24:02.542 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-03-24T22:24:02.542 INFO:teuthology.orchestra.run.smithi018.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-03-24T22:24:02.542 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T22:24:02.543 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T22:24:02.543 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-24T22:24:02.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-24T22:24:02.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-24T22:24:02.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-24T22:24:02.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-24T22:24:02.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-24T22:24:02.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-24T22:24:02.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-24T22:24:02.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-24T22:24:02.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-24T22:24:02.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-24T22:24:02.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-24T22:24:02.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-24T22:24:02.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-24T22:24:02.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-24T22:24:02.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-24T22:24:02.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-24T22:24:02.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-24T22:24:02.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-24T22:24:02.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-24T22:24:02.548 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako-1.0.6-14.el8.noarch 2023-03-24T22:24:02.548 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-24T22:24:02.548 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-24T22:24:02.548 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-24T22:24:02.548 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-24T22:24:02.548 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-24T22:24:02.548 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-24T22:24:02.549 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-24T22:24:02.549 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend-2.6-1.el8.noarch 2023-03-24T22:24:02.549 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-24T22:24:02.549 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-24T22:24:02.549 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-24T22:24:02.549 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-24T22:24:02.549 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-24T22:24:02.550 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-24T22:24:02.550 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.550 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.550 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-24T22:24:02.550 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-24T22:24:02.550 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.550 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-24T22:24:02.550 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-24T22:24:02.551 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-03-24T22:24:02.551 INFO:teuthology.orchestra.run.smithi018.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-24T22:24:02.551 INFO:teuthology.orchestra.run.smithi018.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-24T22:24:02.551 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-24T22:24:02.551 INFO:teuthology.orchestra.run.smithi018.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-24T22:24:02.551 INFO:teuthology.orchestra.run.smithi018.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-24T22:24:02.551 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-24T22:24:02.551 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-24T22:24:02.552 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-24T22:24:02.552 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-24T22:24:02.552 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-24T22:24:02.552 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.552 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.552 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd-2:18.0.0-3124.gc19e8f2a.el8.x86_64 2023-03-24T22:24:02.552 INFO:teuthology.orchestra.run.smithi018.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-24T22:24:02.553 INFO:teuthology.orchestra.run.smithi018.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-24T22:24:02.553 INFO:teuthology.orchestra.run.smithi018.stdout: uuid-1.6.2-43.el8.x86_64 2023-03-24T22:24:02.553 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-03-24T22:24:02.553 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-24T22:24:02.554 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel-1.2.11-19.el8.x86_64 2023-03-24T22:24:02.554 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:02.554 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2023-03-24T22:24:02.712 DEBUG:teuthology.parallel:result is None 2023-03-24T22:24:02.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c19e8f2a497e95f75abc43ec14bc6095e30dd2cb 2023-03-24T22:24:02.852 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-24T22:24:02.876 INFO:teuthology.orchestra.run.smithi018.stdout:18.0.0-3124.gc19e8f2a.el8 2023-03-24T22:24:02.877 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3124.gc19e8f2a.el8 2023-03-24T22:24:02.877 INFO:teuthology.task.install:The correct ceph version 18.0.0-3124.gc19e8f2a is installed. 2023-03-24T22:24:02.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c19e8f2a497e95f75abc43ec14bc6095e30dd2cb 2023-03-24T22:24:03.014 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-24T22:24:03.040 INFO:teuthology.orchestra.run.smithi100.stdout:18.0.0-3124.gc19e8f2a.el8 2023-03-24T22:24:03.040 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3124.gc19e8f2a.el8 2023-03-24T22:24:03.041 INFO:teuthology.task.install:The correct ceph version 18.0.0-3124.gc19e8f2a is installed. 2023-03-24T22:24:03.041 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-03-24T22:24:03.042 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:03.042 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-24T22:24:03.076 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:03.076 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-24T22:24:03.112 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-03-24T22:24:03.112 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:03.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/daemon-helper 2023-03-24T22:24:03.145 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-24T22:24:03.215 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:03.216 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2023-03-24T22:24:03.244 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-24T22:24:03.311 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-03-24T22:24:03.312 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:03.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-24T22:24:03.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-24T22:24:03.410 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:03.411 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-24T22:24:03.437 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-24T22:24:03.505 INFO:teuthology.run_tasks:Running task ceph... 2023-03-24T22:24:03.583 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-03-24T22:24:03.584 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 777 /var/log/ceph 2023-03-24T22:24:03.586 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2023-03-24T22:24:03.613 INFO:tasks.ceph:Disabling ceph logrotate... 2023-03-24T22:24:03.614 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-24T22:24:03.657 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-24T22:24:03.684 INFO:tasks.ceph:Creating extra log directories... 2023-03-24T22:24:03.684 DEBUG:teuthology.orchestra.run.smithi018:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-24T22:24:03.726 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-24T22:24:03.763 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-03-24T22:24:03.764 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-03-24T22:24:03.764 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-03-24T22:24:03.764 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:03.765 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-03-24T22:24:03.812 DEBUG:teuthology.orchestra.run.smithi018:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-03-24T22:24:03.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-03-24T22:24:03.958 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:03.959 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-03-24T22:24:03.976 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-03-24T22:24:04.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-03-24T22:24:04.128 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-03-24T22:24:04.128 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-03-24T22:24:04.129 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2023-03-24_22:00:37-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi/7219596', 'branch': 'wip-pdonnell-testing-20230323.162417', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7219596', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2023-03-24_22:00:37-fs-wip-pdonnell-testing-20230323.162417-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20230323.162417'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20230323.162417', 'sha1': '568504667cd0ede116ce6b46562b9a2881100488'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 55, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20230323.162417', 'suite_path': '/home/teuthworker/src/github.com_batrick_ceph_568504667cd0ede116ce6b46562b9a2881100488/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/batrick/ceph.git', 'suite_sha1': '568504667cd0ede116ce6b46562b9a2881100488', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi018.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvUEChdBPIgigUGWvKnPofuW3lrWraE6RMOEel3fhd2Y92ZHtwvLl6znQDLuVCrofIy9kO98KKu4Ojz5LCBYxusXN6MRCvfLhzCWR15cq3DS2YXK5uYqXC9BFF0OD/r/AswpxXleqzCRi7yCsV1MfAveTDT+f5aw7QcoI6Kh3n6sYU2ZYJPtl2HGvwZO36vi+eZMEeGhkWoqKDD56XHrbgZdGv9jg9qB1IJ7sqop8a+l/rQQdhS3ex2FiXyYuAlyQG98oSLa0Gj134E5vB1io1RkK4ZUeh0q6Hurzd9S9whj6i1rAcVff7VRza6EQ+IlS1cZEhX5UZxVj1pAdbIAWBjuN3E+w6mceL+zyjbu9CpJsySEdxZjTVf0v95ZbK+F3R79RzBYEU5h05OD8PGkn8vk91bHumRqTFkuGq+IJolDUB2mUZkEN2eaI51+MEqpMKhaxiqrk4UuncyshnVwxZ/RgxuIlrNkJKE8Y97O75VhapsA7R5OdEhx1IuE3E3Nc=', 'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMDwhBowsBXOFxn/w06cj4LLga3mXdGLyNAtME6AQlxc92sSuKpqEFuftyXyhctHuPs9QO+N8Ul6IaQBaIPGK4U='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c19e8f2a497e95f75abc43ec14bc6095e30dd2cb'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_data_scan']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'parallel-gzip', 'teuthology_sha1': 'fe9b6e52013911547168fef4a8506325b6c76139', 'timestamp': '2023-03-24_22:00:37', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.4538'} 2023-03-24T22:24:04.130 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-24T22:24:04.132 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-24T22:24:04.181 DEBUG:teuthology.orchestra.run.smithi018:> sudo install -d -m0777 -- /var/run/ceph 2023-03-24T22:24:04.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2023-03-24T22:24:04.247 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:04.248 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/scratch_devs of=/dev/stdout 2023-03-24T22:24:04.277 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-24T22:24:04.278 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_1 2023-03-24T22:24:04.335 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-24T22:24:04.335 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T22:24:04.336 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 387233 Links: 1 2023-03-24T22:24:04.336 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T22:24:04.336 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T22:24:04.336 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2023-03-24 22:23:59.955269403 +0000 2023-03-24T22:24:04.336 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2023-03-24 22:21:13.160776983 +0000 2023-03-24T22:24:04.336 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2023-03-24 22:21:13.160776983 +0000 2023-03-24T22:24:04.336 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: - 2023-03-24T22:24:04.337 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-24T22:24:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2023-03-24T22:24:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2023-03-24T22:24:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 9.8217e-05 s, 5.2 MB/s 2023-03-24T22:24:04.404 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-24T22:24:04.464 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_2 2023-03-24T22:24:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-24T22:24:04.523 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T22:24:04.523 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 381630 Links: 1 2023-03-24T22:24:04.523 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T22:24:04.524 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T22:24:04.524 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2023-03-24 22:23:59.955269403 +0000 2023-03-24T22:24:04.524 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2023-03-24 22:21:13.689770973 +0000 2023-03-24T22:24:04.524 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2023-03-24 22:21:13.689770973 +0000 2023-03-24T22:24:04.524 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: - 2023-03-24T22:24:04.525 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-24T22:24:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2023-03-24T22:24:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2023-03-24T22:24:04.591 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 9.2108e-05 s, 5.6 MB/s 2023-03-24T22:24:04.592 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-24T22:24:04.651 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_3 2023-03-24T22:24:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-24T22:24:04.710 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T22:24:04.710 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 388467 Links: 1 2023-03-24T22:24:04.711 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T22:24:04.711 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T22:24:04.711 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2023-03-24 22:23:59.955269403 +0000 2023-03-24T22:24:04.711 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2023-03-24 22:21:14.233764794 +0000 2023-03-24T22:24:04.711 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2023-03-24 22:21:14.233764794 +0000 2023-03-24T22:24:04.711 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: - 2023-03-24T22:24:04.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-24T22:24:04.777 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2023-03-24T22:24:04.778 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2023-03-24T22:24:04.778 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 9.8499e-05 s, 5.2 MB/s 2023-03-24T22:24:04.779 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-24T22:24:04.840 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_4 2023-03-24T22:24:04.898 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-24T22:24:04.899 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T22:24:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 387381 Links: 1 2023-03-24T22:24:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T22:24:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T22:24:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2023-03-24 22:23:59.955269403 +0000 2023-03-24T22:24:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2023-03-24 22:21:14.786758512 +0000 2023-03-24T22:24:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2023-03-24 22:21:14.786758512 +0000 2023-03-24T22:24:04.901 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: - 2023-03-24T22:24:04.901 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-24T22:24:04.966 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2023-03-24T22:24:04.967 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2023-03-24T22:24:04.967 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000100343 s, 5.1 MB/s 2023-03-24T22:24:04.968 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-24T22:24:05.027 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-24T22:24:05.027 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:05.028 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2023-03-24T22:24:05.050 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-24T22:24:05.050 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2023-03-24T22:24:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-24T22:24:05.111 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T22:24:05.111 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 383946 Links: 1 2023-03-24T22:24:05.112 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T22:24:05.112 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T22:24:05.112 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2023-03-24 22:23:58.162903615 +0000 2023-03-24T22:24:05.112 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2023-03-24 22:21:04.385280745 +0000 2023-03-24T22:24:05.112 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2023-03-24 22:21:04.385280745 +0000 2023-03-24T22:24:05.112 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2023-03-24T22:24:05.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-24T22:24:05.181 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2023-03-24T22:24:05.181 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2023-03-24T22:24:05.182 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000244499 s, 2.1 MB/s 2023-03-24T22:24:05.183 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-24T22:24:05.244 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2023-03-24T22:24:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-24T22:24:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T22:24:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 387416 Links: 1 2023-03-24T22:24:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T22:24:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T22:24:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2023-03-24 22:23:58.162903615 +0000 2023-03-24T22:24:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2023-03-24 22:21:04.899267892 +0000 2023-03-24T22:24:05.306 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2023-03-24 22:21:04.899267892 +0000 2023-03-24T22:24:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2023-03-24T22:24:05.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-24T22:24:05.374 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2023-03-24T22:24:05.375 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2023-03-24T22:24:05.375 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000311748 s, 1.6 MB/s 2023-03-24T22:24:05.376 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-24T22:24:05.437 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2023-03-24T22:24:05.496 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-24T22:24:05.497 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T22:24:05.497 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 390150 Links: 1 2023-03-24T22:24:05.497 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T22:24:05.497 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T22:24:05.497 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2023-03-24 22:23:58.162903615 +0000 2023-03-24T22:24:05.497 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2023-03-24 22:21:05.408255164 +0000 2023-03-24T22:24:05.497 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2023-03-24 22:21:05.408255164 +0000 2023-03-24T22:24:05.498 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2023-03-24T22:24:05.498 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-24T22:24:05.565 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2023-03-24T22:24:05.565 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2023-03-24T22:24:05.565 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000222663 s, 2.3 MB/s 2023-03-24T22:24:05.566 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-24T22:24:05.626 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2023-03-24T22:24:05.688 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-24T22:24:05.689 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-24T22:24:05.689 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 389206 Links: 1 2023-03-24T22:24:05.689 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-24T22:24:05.689 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2023-03-24T22:24:05.689 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2023-03-24 22:23:58.162903615 +0000 2023-03-24T22:24:05.690 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2023-03-24 22:21:05.921242335 +0000 2023-03-24T22:24:05.690 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2023-03-24 22:21:05.921242335 +0000 2023-03-24T22:24:05.690 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2023-03-24T22:24:05.690 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-24T22:24:05.759 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2023-03-24T22:24:05.760 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2023-03-24T22:24:05.760 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000183767 s, 2.8 MB/s 2023-03-24T22:24:05.761 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-24T22:24:05.830 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-24T22:24:05.831 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi018.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-24T22:24:05.831 INFO:tasks.ceph:Generating config... 2023-03-24T22:24:05.832 INFO:tasks.ceph:[client] client mount timeout = 600 2023-03-24T22:24:05.833 INFO:tasks.ceph:[client] debug client = 20 2023-03-24T22:24:05.833 INFO:tasks.ceph:[client] debug ms = 1 2023-03-24T22:24:05.833 INFO:tasks.ceph:[client] pid file = 2023-03-24T22:24:05.834 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-03-24T22:24:05.834 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-03-24T22:24:05.834 INFO:tasks.ceph:[mds] debug mds = 20 2023-03-24T22:24:05.834 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-03-24T22:24:05.834 INFO:tasks.ceph:[mds] debug ms = 1 2023-03-24T22:24:05.835 INFO:tasks.ceph:[mds] mds debug frag = True 2023-03-24T22:24:05.835 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-03-24T22:24:05.835 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-03-24T22:24:05.835 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-03-24T22:24:05.835 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-03-24T22:24:05.836 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-03-24T22:24:05.836 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-03-24T22:24:05.836 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-03-24T22:24:05.836 INFO:tasks.ceph:[mgr] debug ms = 1 2023-03-24T22:24:05.836 INFO:tasks.ceph:[mon] debug mon = 20 2023-03-24T22:24:05.837 INFO:tasks.ceph:[mon] debug ms = 1 2023-03-24T22:24:05.837 INFO:tasks.ceph:[mon] debug paxos = 20 2023-03-24T22:24:05.837 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-03-24T22:24:05.837 INFO:tasks.ceph:[osd] bdev async discard = True 2023-03-24T22:24:05.838 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-03-24T22:24:05.838 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-03-24T22:24:05.838 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-03-24T22:24:05.838 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-03-24T22:24:05.838 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-03-24T22:24:05.839 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-03-24T22:24:05.839 INFO:tasks.ceph:[osd] debug ms = 1 2023-03-24T22:24:05.839 INFO:tasks.ceph:[osd] debug osd = 20 2023-03-24T22:24:05.839 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-03-24T22:24:05.840 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-03-24T22:24:05.840 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-03-24T22:24:05.840 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-03-24T22:24:05.840 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-03-24T22:24:05.841 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-03-24T22:24:05.841 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-03-24T22:24:05.841 INFO:tasks.ceph:Setting up mon.a... 2023-03-24T22:24:05.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-03-24T22:24:05.979 INFO:teuthology.orchestra.run.smithi018.stdout:creating /etc/ceph/ceph.keyring 2023-03-24T22:24:05.982 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-03-24T22:24:06.198 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-24T22:24:06.230 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.18'), ('mon.b', '172.21.15.100'), ('mon.c', '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]')] 2023-03-24T22:24:06.231 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.18,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'false', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-03-24T22:24:06.232 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:06.232 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-03-24T22:24:06.290 DEBUG:teuthology.orchestra.run.smithi018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.18 --add b 172.21.15.100 --addv c '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:06.385 INFO:teuthology.orchestra.run.smithi018.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:06.385 INFO:teuthology.orchestra.run.smithi018.stdout:monmaptool: generated fsid 67a1ac4c-1568-462e-b9b7-d22a9a29a86a 2023-03-24T22:24:06.386 INFO:teuthology.orchestra.run.smithi018.stdout:setting min_mon_release = pacific 2023-03-24T22:24:06.386 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 0 2023-03-24T22:24:06.386 INFO:teuthology.orchestra.run.smithi018.stdout:fsid 67a1ac4c-1568-462e-b9b7-d22a9a29a86a 2023-03-24T22:24:06.386 INFO:teuthology.orchestra.run.smithi018.stdout:last_changed 2023-03-24T22:24:06.385490+0000 2023-03-24T22:24:06.386 INFO:teuthology.orchestra.run.smithi018.stdout:created 2023-03-24T22:24:06.385490+0000 2023-03-24T22:24:06.386 INFO:teuthology.orchestra.run.smithi018.stdout:min_mon_release 16 (pacific) 2023-03-24T22:24:06.387 INFO:teuthology.orchestra.run.smithi018.stdout:election_strategy: 1 2023-03-24T22:24:06.387 INFO:teuthology.orchestra.run.smithi018.stdout:0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.a 2023-03-24T22:24:06.387 INFO:teuthology.orchestra.run.smithi018.stdout:1: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.b 2023-03-24T22:24:06.387 INFO:teuthology.orchestra.run.smithi018.stdout:2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2023-03-24T22:24:06.387 INFO:teuthology.orchestra.run.smithi018.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-03-24T22:24:06.388 DEBUG:teuthology.orchestra.run.smithi018:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-03-24T22:24:06.408 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 67a1ac4c-1568-462e-b9b7-d22a9a29a86a... 2023-03-24T22:24:06.409 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-24T22:24:06.451 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-24T22:24:06.497 INFO:teuthology.orchestra.run.smithi018.stdout:[global] 2023-03-24T22:24:06.497 INFO:teuthology.orchestra.run.smithi018.stdout: chdir = "" 2023-03-24T22:24:06.497 INFO:teuthology.orchestra.run.smithi018.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-24T22:24:06.498 INFO:teuthology.orchestra.run.smithi018.stdout: auth supported = cephx 2023-03-24T22:24:06.498 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.498 INFO:teuthology.orchestra.run.smithi018.stdout: filestore xattr use omap = true 2023-03-24T22:24:06.498 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.498 INFO:teuthology.orchestra.run.smithi018.stdout: mon clock drift allowed = 1.000 2023-03-24T22:24:06.498 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.498 INFO:teuthology.orchestra.run.smithi018.stdout: osd crush chooseleaf type = 0 2023-03-24T22:24:06.498 INFO:teuthology.orchestra.run.smithi018.stdout: auth debug = true 2023-03-24T22:24:06.499 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.499 INFO:teuthology.orchestra.run.smithi018.stdout: ms die on old message = true 2023-03-24T22:24:06.499 INFO:teuthology.orchestra.run.smithi018.stdout: ms die on bug = true 2023-03-24T22:24:06.499 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.499 INFO:teuthology.orchestra.run.smithi018.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-24T22:24:06.499 INFO:teuthology.orchestra.run.smithi018.stdout: mon pg warn max object skew = 0 2023-03-24T22:24:06.499 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.499 INFO:teuthology.orchestra.run.smithi018.stdout: # disable pg_autoscaler by default for new pools 2023-03-24T22:24:06.500 INFO:teuthology.orchestra.run.smithi018.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-24T22:24:06.500 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.500 INFO:teuthology.orchestra.run.smithi018.stdout: osd pool default size = 2 2023-03-24T22:24:06.500 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.500 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd allow primary affinity = true 2023-03-24T22:24:06.500 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd allow pg remap = true 2023-03-24T22:24:06.500 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on legacy crush tunables = false 2023-03-24T22:24:06.501 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on crush straw calc version zero = false 2023-03-24T22:24:06.501 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on no sortbitwise = false 2023-03-24T22:24:06.501 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on osd down out interval zero = false 2023-03-24T22:24:06.501 INFO:teuthology.orchestra.run.smithi018.stdout: mon warn on too few osds = false 2023-03-24T22:24:06.501 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-24T22:24:06.501 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-24T22:24:06.501 INFO:teuthology.orchestra.run.smithi018.stdout: mon_allow_pool_size_one = true 2023-03-24T22:24:06.501 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.502 INFO:teuthology.orchestra.run.smithi018.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-24T22:24:06.502 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.502 INFO:teuthology.orchestra.run.smithi018.stdout: osd default data pool replay window = 5 2023-03-24T22:24:06.502 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.502 INFO:teuthology.orchestra.run.smithi018.stdout: mon allow pool delete = true 2023-03-24T22:24:06.502 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.502 INFO:teuthology.orchestra.run.smithi018.stdout: mon cluster log file level = debug 2023-03-24T22:24:06.502 INFO:teuthology.orchestra.run.smithi018.stdout: debug asserts on shutdown = true 2023-03-24T22:24:06.503 INFO:teuthology.orchestra.run.smithi018.stdout: mon health detail to clog = false 2023-03-24T22:24:06.503 INFO:teuthology.orchestra.run.smithi018.stdout: mon host = "172.21.15.18,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2023-03-24T22:24:06.503 INFO:teuthology.orchestra.run.smithi018.stdout: fsid = 67a1ac4c-1568-462e-b9b7-d22a9a29a86a 2023-03-24T22:24:06.503 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.503 INFO:teuthology.orchestra.run.smithi018.stdout:[osd] 2023-03-24T22:24:06.503 INFO:teuthology.orchestra.run.smithi018.stdout: osd journal size = 100 2023-03-24T22:24:06.503 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.504 INFO:teuthology.orchestra.run.smithi018.stdout: osd scrub load threshold = 5.0 2023-03-24T22:24:06.504 INFO:teuthology.orchestra.run.smithi018.stdout: osd scrub max interval = 600 2023-03-24T22:24:06.504 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.504 INFO:teuthology.orchestra.run.smithi018.stdout: osd recover clone overlap = true 2023-03-24T22:24:06.504 INFO:teuthology.orchestra.run.smithi018.stdout: osd recovery max chunk = 1048576 2023-03-24T22:24:06.504 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.504 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug shutdown = true 2023-03-24T22:24:06.504 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug op order = false 2023-03-24T22:24:06.505 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug verify stray on activate = true 2023-03-24T22:24:06.505 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.505 INFO:teuthology.orchestra.run.smithi018.stdout: osd open classes on start = true 2023-03-24T22:24:06.505 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug pg log writeout = true 2023-03-24T22:24:06.505 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.505 INFO:teuthology.orchestra.run.smithi018.stdout: osd deep scrub update digest min age = 30 2023-03-24T22:24:06.505 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.505 INFO:teuthology.orchestra.run.smithi018.stdout: osd map max advance = 10 2023-03-24T22:24:06.506 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.506 INFO:teuthology.orchestra.run.smithi018.stdout: journal zero on create = true 2023-03-24T22:24:06.506 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.506 INFO:teuthology.orchestra.run.smithi018.stdout: filestore ondisk finisher threads = 3 2023-03-24T22:24:06.506 INFO:teuthology.orchestra.run.smithi018.stdout: filestore apply finisher threads = 3 2023-03-24T22:24:06.506 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.506 INFO:teuthology.orchestra.run.smithi018.stdout: bdev debug aio = true 2023-03-24T22:24:06.506 INFO:teuthology.orchestra.run.smithi018.stdout: osd debug misdirected ops = true 2023-03-24T22:24:06.507 INFO:teuthology.orchestra.run.smithi018.stdout: bdev async discard = True 2023-03-24T22:24:06.507 INFO:teuthology.orchestra.run.smithi018.stdout: bdev enable discard = True 2023-03-24T22:24:06.507 INFO:teuthology.orchestra.run.smithi018.stdout: bluestore allocator = bitmap 2023-03-24T22:24:06.507 INFO:teuthology.orchestra.run.smithi018.stdout: bluestore block size = 96636764160 2023-03-24T22:24:06.507 INFO:teuthology.orchestra.run.smithi018.stdout: bluestore fsck on mount = True 2023-03-24T22:24:06.507 INFO:teuthology.orchestra.run.smithi018.stdout: debug bluefs = 1/20 2023-03-24T22:24:06.507 INFO:teuthology.orchestra.run.smithi018.stdout: debug bluestore = 1/20 2023-03-24T22:24:06.507 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2023-03-24T22:24:06.508 INFO:teuthology.orchestra.run.smithi018.stdout: debug osd = 20 2023-03-24T22:24:06.508 INFO:teuthology.orchestra.run.smithi018.stdout: debug rocksdb = 4/10 2023-03-24T22:24:06.508 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd backfillfull_ratio = 0.85 2023-03-24T22:24:06.508 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd full ratio = 0.9 2023-03-24T22:24:06.508 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd nearfull ratio = 0.8 2023-03-24T22:24:06.508 INFO:teuthology.orchestra.run.smithi018.stdout: osd failsafe full ratio = 0.95 2023-03-24T22:24:06.508 INFO:teuthology.orchestra.run.smithi018.stdout: osd objectstore = bluestore 2023-03-24T22:24:06.508 INFO:teuthology.orchestra.run.smithi018.stdout: osd op complaint time = 180 2023-03-24T22:24:06.509 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.509 INFO:teuthology.orchestra.run.smithi018.stdout:[mgr] 2023-03-24T22:24:06.509 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2023-03-24T22:24:06.509 INFO:teuthology.orchestra.run.smithi018.stdout: debug mgr = 20 2023-03-24T22:24:06.509 INFO:teuthology.orchestra.run.smithi018.stdout: debug mon = 20 2023-03-24T22:24:06.509 INFO:teuthology.orchestra.run.smithi018.stdout: debug auth = 20 2023-03-24T22:24:06.509 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min pgs per osd = 4 2023-03-24T22:24:06.510 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min bytes per osd = 10 2023-03-24T22:24:06.510 INFO:teuthology.orchestra.run.smithi018.stdout: mgr/telemetry/nag = false 2023-03-24T22:24:06.510 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.510 INFO:teuthology.orchestra.run.smithi018.stdout:[mon] 2023-03-24T22:24:06.510 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2023-03-24T22:24:06.510 INFO:teuthology.orchestra.run.smithi018.stdout: debug mon = 20 2023-03-24T22:24:06.510 INFO:teuthology.orchestra.run.smithi018.stdout: debug paxos = 20 2023-03-24T22:24:06.510 INFO:teuthology.orchestra.run.smithi018.stdout: debug auth = 20 2023-03-24T22:24:06.511 INFO:teuthology.orchestra.run.smithi018.stdout: mon data avail warn = 5 2023-03-24T22:24:06.511 INFO:teuthology.orchestra.run.smithi018.stdout: mon mgr mkfs grace = 240 2023-03-24T22:24:06.511 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min pgs per osd = 4 2023-03-24T22:24:06.511 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd reporter subtree level = osd 2023-03-24T22:24:06.511 INFO:teuthology.orchestra.run.smithi018.stdout: mon osd prime pg temp = true 2023-03-24T22:24:06.511 INFO:teuthology.orchestra.run.smithi018.stdout: mon reweight min bytes per osd = 10 2023-03-24T22:24:06.511 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.511 INFO:teuthology.orchestra.run.smithi018.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-24T22:24:06.512 INFO:teuthology.orchestra.run.smithi018.stdout: auth mon ticket ttl = 660 # 11m 2023-03-24T22:24:06.512 INFO:teuthology.orchestra.run.smithi018.stdout: auth service ticket ttl = 240 # 4m 2023-03-24T22:24:06.512 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.512 INFO:teuthology.orchestra.run.smithi018.stdout: # don't complain about insecure global_id in the test suite 2023-03-24T22:24:06.512 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-24T22:24:06.512 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-24T22:24:06.512 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.512 INFO:teuthology.orchestra.run.smithi018.stdout: # 1m isn't quite enough 2023-03-24T22:24:06.513 INFO:teuthology.orchestra.run.smithi018.stdout: mon_down_mkfs_grace = 2m 2023-03-24T22:24:06.513 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.513 INFO:teuthology.orchestra.run.smithi018.stdout: mon_warn_on_filestore_osds = false 2023-03-24T22:24:06.513 INFO:teuthology.orchestra.run.smithi018.stdout: mon op complaint time = 120 2023-03-24T22:24:06.513 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:06.513 INFO:teuthology.orchestra.run.smithi018.stdout:[client] 2023-03-24T22:24:06.513 INFO:teuthology.orchestra.run.smithi018.stdout: rgw cache enabled = true 2023-03-24T22:24:06.513 INFO:teuthology.orchestra.run.smithi018.stdout: rgw enable ops log = true 2023-03-24T22:24:06.514 INFO:teuthology.orchestra.run.smithi018.stdout: rgw enable usage log = true 2023-03-24T22:24:06.514 INFO:teuthology.orchestra.run.smithi018.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-24T22:24:06.514 INFO:teuthology.orchestra.run.smithi018.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-24T22:24:06.514 INFO:teuthology.orchestra.run.smithi018.stdout: client mount timeout = 600 2023-03-24T22:24:06.514 INFO:teuthology.orchestra.run.smithi018.stdout: debug client = 20 2023-03-24T22:24:06.514 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2023-03-24T22:24:06.514 INFO:teuthology.orchestra.run.smithi018.stdout: pid file = "" 2023-03-24T22:24:06.514 INFO:teuthology.orchestra.run.smithi018.stdout: rados mon op timeout = 900 2023-03-24T22:24:06.515 INFO:teuthology.orchestra.run.smithi018.stdout: rados osd op timeout = 900 2023-03-24T22:24:06.515 INFO:teuthology.orchestra.run.smithi018.stdout:[mon.a] 2023-03-24T22:24:06.515 INFO:teuthology.orchestra.run.smithi018.stdout:[mon.b] 2023-03-24T22:24:06.515 INFO:teuthology.orchestra.run.smithi018.stdout:[mon.c] 2023-03-24T22:24:06.515 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.a] 2023-03-24T22:24:06.515 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.c] 2023-03-24T22:24:06.515 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.b] 2023-03-24T22:24:06.515 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.d] 2023-03-24T22:24:06.516 INFO:teuthology.orchestra.run.smithi018.stdout:[mds] 2023-03-24T22:24:06.516 INFO:teuthology.orchestra.run.smithi018.stdout: debug mds = 20 2023-03-24T22:24:06.516 INFO:teuthology.orchestra.run.smithi018.stdout: debug mds balancer = 20 2023-03-24T22:24:06.516 INFO:teuthology.orchestra.run.smithi018.stdout: debug ms = 1 2023-03-24T22:24:06.516 INFO:teuthology.orchestra.run.smithi018.stdout: mds debug frag = True 2023-03-24T22:24:06.516 INFO:teuthology.orchestra.run.smithi018.stdout: mds debug scatterstat = True 2023-03-24T22:24:06.516 INFO:teuthology.orchestra.run.smithi018.stdout: mds op complaint time = 180 2023-03-24T22:24:06.516 INFO:teuthology.orchestra.run.smithi018.stdout: mds verify scatter = True 2023-03-24T22:24:06.517 INFO:teuthology.orchestra.run.smithi018.stdout: osd op complaint time = 180 2023-03-24T22:24:06.517 INFO:teuthology.orchestra.run.smithi018.stdout: rados mon op timeout = 900 2023-03-24T22:24:06.517 INFO:teuthology.orchestra.run.smithi018.stdout: rados osd op timeout = 900 2023-03-24T22:24:06.518 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2023-03-24T22:24:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2023-03-24T22:24:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-24T22:24:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2023-03-24T22:24:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2023-03-24T22:24:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2023-03-24T22:24:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2023-03-24T22:24:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2023-03-24T22:24:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2023-03-24T22:24:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2023-03-24T22:24:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-24T22:24:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2023-03-24T22:24:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2023-03-24T22:24:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-24T22:24:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2023-03-24T22:24:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2023-03-24T22:24:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2023-03-24T22:24:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2023-03-24T22:24:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2023-03-24T22:24:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2023-03-24T22:24:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2023-03-24T22:24:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2023-03-24T22:24:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-24T22:24:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-24T22:24:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2023-03-24T22:24:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-24T22:24:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2023-03-24T22:24:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2023-03-24T22:24:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2023-03-24T22:24:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2023-03-24T22:24:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2023-03-24T22:24:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.18,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2023-03-24T22:24:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 67a1ac4c-1568-462e-b9b7-d22a9a29a86a 2023-03-24T22:24:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.524 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2023-03-24T22:24:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2023-03-24T22:24:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2023-03-24T22:24:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2023-03-24T22:24:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2023-03-24T22:24:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2023-03-24T22:24:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2023-03-24T22:24:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = false 2023-03-24T22:24:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2023-03-24T22:24:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2023-03-24T22:24:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2023-03-24T22:24:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2023-03-24T22:24:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2023-03-24T22:24:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2023-03-24T22:24:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2023-03-24T22:24:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2023-03-24T22:24:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2023-03-24T22:24:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2023-03-24T22:24:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2023-03-24T22:24:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2023-03-24T22:24:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2023-03-24T22:24:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2023-03-24T22:24:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2023-03-24T22:24:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2023-03-24T22:24:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2023-03-24T22:24:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2023-03-24T22:24:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2023-03-24T22:24:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2023-03-24T22:24:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2023-03-24T22:24:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2023-03-24T22:24:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2023-03-24T22:24:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2023-03-24T22:24:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2023-03-24T22:24:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2023-03-24T22:24:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.530 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2023-03-24T22:24:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2023-03-24T22:24:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2023-03-24T22:24:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2023-03-24T22:24:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2023-03-24T22:24:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2023-03-24T22:24:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2023-03-24T22:24:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2023-03-24T22:24:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.531 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2023-03-24T22:24:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2023-03-24T22:24:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2023-03-24T22:24:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2023-03-24T22:24:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2023-03-24T22:24:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2023-03-24T22:24:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2023-03-24T22:24:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2023-03-24T22:24:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2023-03-24T22:24:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2023-03-24T22:24:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2023-03-24T22:24:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-24T22:24:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2023-03-24T22:24:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2023-03-24T22:24:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2023-03-24T22:24:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-24T22:24:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-24T22:24:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2023-03-24T22:24:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2023-03-24T22:24:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2023-03-24T22:24:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2023-03-24T22:24:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-03-24T22:24:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2023-03-24T22:24:06.535 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2023-03-24T22:24:06.535 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2023-03-24T22:24:06.535 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2023-03-24T22:24:06.535 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-24T22:24:06.535 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-24T22:24:06.535 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2023-03-24T22:24:06.535 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2023-03-24T22:24:06.535 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2023-03-24T22:24:06.536 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2023-03-24T22:24:06.536 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2023-03-24T22:24:06.536 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2023-03-24T22:24:06.536 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2023-03-24T22:24:06.536 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2023-03-24T22:24:06.536 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2023-03-24T22:24:06.536 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2023-03-24T22:24:06.537 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2023-03-24T22:24:06.537 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2023-03-24T22:24:06.537 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2023-03-24T22:24:06.537 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2023-03-24T22:24:06.537 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2023-03-24T22:24:06.537 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2023-03-24T22:24:06.537 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2023-03-24T22:24:06.537 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2023-03-24T22:24:06.538 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = True 2023-03-24T22:24:06.538 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2023-03-24T22:24:06.538 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = True 2023-03-24T22:24:06.538 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2023-03-24T22:24:06.538 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2023-03-24T22:24:06.538 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2023-03-24T22:24:06.538 INFO:tasks.ceph:Creating admin key on mon.a... 2023-03-24T22:24:06.539 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-03-24T22:24:06.618 INFO:tasks.ceph:Copying monmap to all nodes... 2023-03-24T22:24:06.618 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:06.618 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-03-24T22:24:06.636 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:06.637 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-03-24T22:24:06.694 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi018.front.sepia.ceph.com 2023-03-24T22:24:06.695 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:06.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-24T22:24:06.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-24T22:24:06.772 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:06.773 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:06.831 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2023-03-24T22:24:06.831 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:06.831 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-24T22:24:06.832 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-24T22:24:06.870 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:06.871 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:06.929 INFO:tasks.ceph:Setting up mon nodes... 2023-03-24T22:24:06.930 INFO:tasks.ceph:Setting up mgr nodes... 2023-03-24T22:24:06.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-03-24T22:24:06.985 INFO:teuthology.orchestra.run.smithi018.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-03-24T22:24:06.988 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-03-24T22:24:07.076 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-03-24T22:24:07.079 INFO:tasks.ceph:Setting up mds nodes... 2023-03-24T22:24:07.079 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-03-24T22:24:07.135 INFO:teuthology.orchestra.run.smithi018.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-03-24T22:24:07.137 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-03-24T22:24:07.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-03-24T22:24:07.261 INFO:teuthology.orchestra.run.smithi018.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-03-24T22:24:07.263 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-03-24T22:24:07.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-03-24T22:24:07.346 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-03-24T22:24:07.348 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-03-24T22:24:07.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-03-24T22:24:07.471 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-03-24T22:24:07.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-03-24T22:24:07.500 INFO:tasks.ceph_client:Setting up client nodes... 2023-03-24T22:24:07.500 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-03-24T22:24:07.546 INFO:teuthology.orchestra.run.smithi018.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-03-24T22:24:07.558 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-03-24T22:24:07.605 INFO:teuthology.orchestra.run.smithi018.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-03-24T22:24:07.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-03-24T22:24:07.661 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-03-24T22:24:07.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-03-24T22:24:07.716 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-03-24T22:24:07.728 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-03-24T22:24:07.729 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi018.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-24T22:24:07.729 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-03-24T22:24:07.758 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-24T22:24:07.758 INFO:tasks.ceph:role: osd.0 2023-03-24T22:24:07.758 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi018.front.sepia.ceph.com 2023-03-24T22:24:07.759 DEBUG:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-24T22:24:07.825 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T22:24:07.826 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T22:24:07.826 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T22:24:07.826 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T22:24:07.826 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T22:24:07.826 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2023-03-24T22:24:07.826 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T22:24:07.827 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T22:24:07.827 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T22:24:07.827 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T22:24:07.848 INFO:teuthology.orchestra.run.smithi018.stdout:Discarding blocks...Done. 2023-03-24T22:24:07.850 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2023-03-24T22:24:07.851 DEBUG:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-03-24T22:24:07.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-03-24T22:24:08.007 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-03-24T22:24:08.076 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-24T22:24:08.076 INFO:tasks.ceph:role: osd.1 2023-03-24T22:24:08.077 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi018.front.sepia.ceph.com 2023-03-24T22:24:08.077 DEBUG:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-24T22:24:08.145 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T22:24:08.145 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T22:24:08.145 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T22:24:08.145 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T22:24:08.145 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T22:24:08.146 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2023-03-24T22:24:08.146 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T22:24:08.146 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T22:24:08.146 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T22:24:08.146 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T22:24:08.169 INFO:teuthology.orchestra.run.smithi018.stdout:Discarding blocks...Done. 2023-03-24T22:24:08.171 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2023-03-24T22:24:08.171 DEBUG:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-03-24T22:24:08.253 DEBUG:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-03-24T22:24:08.325 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-03-24T22:24:08.394 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-24T22:24:08.394 INFO:tasks.ceph:role: osd.2 2023-03-24T22:24:08.395 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi018.front.sepia.ceph.com 2023-03-24T22:24:08.395 DEBUG:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-24T22:24:08.464 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T22:24:08.465 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T22:24:08.465 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T22:24:08.465 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T22:24:08.465 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T22:24:08.465 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2023-03-24T22:24:08.465 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T22:24:08.466 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T22:24:08.466 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T22:24:08.466 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T22:24:08.487 INFO:teuthology.orchestra.run.smithi018.stdout:Discarding blocks...Done. 2023-03-24T22:24:08.489 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2023-03-24T22:24:08.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-03-24T22:24:08.589 DEBUG:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-03-24T22:24:08.623 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-03-24T22:24:08.692 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-24T22:24:08.692 INFO:tasks.ceph:role: osd.3 2023-03-24T22:24:08.693 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi018.front.sepia.ceph.com 2023-03-24T22:24:08.693 DEBUG:teuthology.orchestra.run.smithi018:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-24T22:24:08.760 INFO:teuthology.orchestra.run.smithi018.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T22:24:08.760 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T22:24:08.761 INFO:teuthology.orchestra.run.smithi018.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T22:24:08.761 INFO:teuthology.orchestra.run.smithi018.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T22:24:08.761 INFO:teuthology.orchestra.run.smithi018.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T22:24:08.761 INFO:teuthology.orchestra.run.smithi018.stdout: = sunit=0 swidth=0 blks 2023-03-24T22:24:08.761 INFO:teuthology.orchestra.run.smithi018.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T22:24:08.761 INFO:teuthology.orchestra.run.smithi018.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T22:24:08.761 INFO:teuthology.orchestra.run.smithi018.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T22:24:08.762 INFO:teuthology.orchestra.run.smithi018.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T22:24:08.782 INFO:teuthology.orchestra.run.smithi018.stdout:Discarding blocks...Done. 2023-03-24T22:24:08.784 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi018.front.sepia.ceph.com -o noatime 2023-03-24T22:24:08.784 DEBUG:teuthology.orchestra.run.smithi018:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-03-24T22:24:08.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-03-24T22:24:08.919 DEBUG:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:09.018 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:09.016+0000 7f87e7cdd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-03-24T22:24:09.018 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:09.016+0000 7f87e7cdd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-03-24T22:24:09.019 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:09.017+0000 7f87e7cdd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-24T22:24:09.019 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:09.017+0000 7f87e7cdd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-24T22:24:09.019 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:09.017+0000 7f87e7cdd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-24T22:24:09.019 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:09.017+0000 7f87e7cdd640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-03-24T22:24:11.871 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-03-24T22:24:11.906 DEBUG:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:12.003+0000 7f2256971640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-03-24T22:24:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:12.004+0000 7f2256971640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-03-24T22:24:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:12.004+0000 7f2256971640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-24T22:24:12.006 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:12.004+0000 7f2256971640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-24T22:24:12.006 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:12.004+0000 7f2256971640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-24T22:24:12.006 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:12.004+0000 7f2256971640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-03-24T22:24:14.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-03-24T22:24:14.893 DEBUG:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:14.989 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:14.987+0000 7f7d40fa6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-03-24T22:24:14.989 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:14.987+0000 7f7d40fa6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-03-24T22:24:14.989 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:14.987+0000 7f7d40fa6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-24T22:24:14.990 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:14.987+0000 7f7d40fa6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-24T22:24:14.990 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:14.987+0000 7f7d40fa6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-24T22:24:14.990 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:14.987+0000 7f7d40fa6640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-03-24T22:24:17.837 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-03-24T22:24:17.871 DEBUG:teuthology.orchestra.run.smithi018:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:17.975 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:17.974+0000 7f3aaccf3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-03-24T22:24:17.976 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:17.974+0000 7f3aaccf3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-03-24T22:24:17.976 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:17.974+0000 7f3aaccf3640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-24T22:24:17.976 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:17.974+0000 7f3aaccf3640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-24T22:24:17.976 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:17.974+0000 7f3aaccf3640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-24T22:24:17.976 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:17.974+0000 7f3aaccf3640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-03-24T22:24:20.827 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-03-24T22:24:20.862 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-03-24T22:24:20.895 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-24T22:24:20.895 INFO:tasks.ceph:role: osd.4 2023-03-24T22:24:20.895 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2023-03-24T22:24:20.896 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-24T22:24:20.973 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T22:24:20.974 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T22:24:20.974 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T22:24:20.974 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T22:24:20.974 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T22:24:20.974 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2023-03-24T22:24:20.974 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T22:24:20.974 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T22:24:20.975 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T22:24:20.975 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T22:24:21.405 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2023-03-24T22:24:21.406 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2023-03-24T22:24:21.407 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-03-24T22:24:21.594 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-03-24T22:24:21.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-03-24T22:24:21.705 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-24T22:24:21.705 INFO:tasks.ceph:role: osd.5 2023-03-24T22:24:21.705 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2023-03-24T22:24:21.706 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-24T22:24:21.783 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T22:24:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T22:24:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T22:24:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T22:24:21.784 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T22:24:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2023-03-24T22:24:21.784 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T22:24:21.784 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T22:24:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T22:24:21.784 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T22:24:22.197 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2023-03-24T22:24:22.199 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2023-03-24T22:24:22.199 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-03-24T22:24:22.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-03-24T22:24:22.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-03-24T22:24:22.412 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-24T22:24:22.412 INFO:tasks.ceph:role: osd.6 2023-03-24T22:24:22.412 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2023-03-24T22:24:22.413 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-24T22:24:22.487 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T22:24:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T22:24:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T22:24:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T22:24:22.488 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T22:24:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2023-03-24T22:24:22.489 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T22:24:22.489 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T22:24:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T22:24:22.489 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T22:24:22.943 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2023-03-24T22:24:22.945 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2023-03-24T22:24:22.945 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-03-24T22:24:23.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-03-24T22:24:23.148 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-03-24T22:24:23.215 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-24T22:24:23.215 INFO:tasks.ceph:role: osd.7 2023-03-24T22:24:23.215 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2023-03-24T22:24:23.216 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-24T22:24:23.289 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-24T22:24:23.290 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-24T22:24:23.290 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-24T22:24:23.290 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-24T22:24:23.290 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-24T22:24:23.290 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2023-03-24T22:24:23.290 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-24T22:24:23.290 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-24T22:24:23.291 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-24T22:24:23.291 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-24T22:24:23.724 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2023-03-24T22:24:23.726 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2023-03-24T22:24:23.726 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-03-24T22:24:23.894 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-03-24T22:24:23.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:24.175 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:24.174+0000 7f1846cd3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-03-24T22:24:24.176 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:24.174+0000 7f1846cd3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-03-24T22:24:24.176 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:24.174+0000 7f1846cd3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-24T22:24:24.176 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:24.174+0000 7f1846cd3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-24T22:24:24.176 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:24.174+0000 7f1846cd3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-24T22:24:24.177 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:24.175+0000 7f1846cd3640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-03-24T22:24:27.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-03-24T22:24:27.056 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:27.148 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:27.148+0000 7ff709439640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-03-24T22:24:27.149 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:27.148+0000 7ff709439640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-03-24T22:24:27.149 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:27.148+0000 7ff709439640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-24T22:24:27.149 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:27.148+0000 7ff709439640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-24T22:24:27.150 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:27.148+0000 7ff709439640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-24T22:24:27.150 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:27.149+0000 7ff709439640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-03-24T22:24:30.004 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-03-24T22:24:30.036 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:30.128 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:30.128+0000 7fd1631aa640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-03-24T22:24:30.128 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:30.128+0000 7fd1631aa640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-03-24T22:24:30.128 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:30.128+0000 7fd1631aa640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-24T22:24:30.129 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:30.128+0000 7fd1631aa640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-24T22:24:30.129 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:30.128+0000 7fd1631aa640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-24T22:24:30.129 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:30.128+0000 7fd1631aa640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-03-24T22:24:32.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-03-24T22:24:33.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:33.101+0000 7f47ce10f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-03-24T22:24:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:33.101+0000 7f47ce10f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-03-24T22:24:33.102 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:33.101+0000 7f47ce10f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-24T22:24:33.102 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:33.101+0000 7f47ce10f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-24T22:24:33.102 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:33.101+0000 7f47ce10f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-24T22:24:33.102 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:33.101+0000 7f47ce10f640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-03-24T22:24:34.132 DEBUG:teuthology.orchestra.run.smithi018:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-24T22:24:34.135 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-24T22:24:35.948 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-03-24T22:24:35.978 INFO:tasks.ceph:Reading keys from all nodes... 2023-03-24T22:24:35.979 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:35.979 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-03-24T22:24:36.011 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:36.011 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-03-24T22:24:36.082 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:36.083 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-03-24T22:24:36.153 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:36.153 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-03-24T22:24:36.222 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:36.222 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-03-24T22:24:36.291 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:36.292 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-03-24T22:24:36.360 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:36.360 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-03-24T22:24:36.429 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:36.429 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-03-24T22:24:36.462 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:36.462 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-03-24T22:24:36.534 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:36.534 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-03-24T22:24:36.603 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:36.604 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-03-24T22:24:36.673 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:36.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-03-24T22:24:36.741 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:36.742 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-03-24T22:24:36.810 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:36.810 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-03-24T22:24:36.877 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:36.877 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-03-24T22:24:36.896 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:36.897 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-03-24T22:24:36.956 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:36.956 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-03-24T22:24:36.974 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:36.974 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-03-24T22:24:37.031 INFO:tasks.ceph:Adding keys to all mons... 2023-03-24T22:24:37.032 DEBUG:teuthology.orchestra.run.smithi018:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-24T22:24:37.034 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-24T22:24:37.073 INFO:teuthology.orchestra.run.smithi018.stdout:[mgr.y] 2023-03-24T22:24:37.074 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAGIx5keIDFOhAA/bV/D8JE9IBHuz+eYIZgFA== 2023-03-24T22:24:37.074 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.a] 2023-03-24T22:24:37.074 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAHIx5kpcYXCBAA4sQYlV4RRHuORihYQIjR4A== 2023-03-24T22:24:37.074 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.c] 2023-03-24T22:24:37.074 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAHIx5kjSuUDxAAZvFferwYHJ1W5pVNVWqBbw== 2023-03-24T22:24:37.075 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.0] 2023-03-24T22:24:37.075 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAJIx5kgDkPARAAG7wjYfoBURGnp8+Gvs60YQ== 2023-03-24T22:24:37.075 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.1] 2023-03-24T22:24:37.075 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAMIx5kRqVKABAAz12j6eNrX0o/iBr/9cXHTQ== 2023-03-24T22:24:37.075 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.2] 2023-03-24T22:24:37.075 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAOIx5kDNDoOhAAVrzpJcQjEUAnQLepJ/BGBQ== 2023-03-24T22:24:37.075 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.3] 2023-03-24T22:24:37.075 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQARIx5kl8IiOhAAhNqbAoFbSCuY/+fmQ++wlQ== 2023-03-24T22:24:37.076 INFO:teuthology.orchestra.run.smithi018.stdout:[mgr.x] 2023-03-24T22:24:37.076 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAHIx5kyKeWBBAAfr+oDF6xm8MNNGwiAkA1RA== 2023-03-24T22:24:37.076 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.b] 2023-03-24T22:24:37.076 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAHIx5kXw+qFBAAkAs+SPbIoHheegrb+Do3gQ== 2023-03-24T22:24:37.076 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.d] 2023-03-24T22:24:37.076 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAHIx5kCusVHBAABLQhtx1pIytL9ukBSy/avA== 2023-03-24T22:24:37.076 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.4] 2023-03-24T22:24:37.077 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAYIx5kXJ15ChAAgJHDb9CgTB8yrinbU8H8Tw== 2023-03-24T22:24:37.077 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.5] 2023-03-24T22:24:37.077 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAbIx5kRF3dCBAA3RymeE8iQ48Uqpthph1VVw== 2023-03-24T22:24:37.077 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.6] 2023-03-24T22:24:37.077 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAeIx5kPzmlBxAAuBlbwACPmVOgjujWwYpPSg== 2023-03-24T22:24:37.077 INFO:teuthology.orchestra.run.smithi018.stdout:[osd.7] 2023-03-24T22:24:37.077 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAhIx5k340KBhAA3z9NIrYnowFeDaoyO1c3ZA== 2023-03-24T22:24:37.077 INFO:teuthology.orchestra.run.smithi018.stdout:[client.0] 2023-03-24T22:24:37.078 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAHIx5k5AKTIBAAQQlDxeS1hC0KlxwRKUVpbA== 2023-03-24T22:24:37.078 INFO:teuthology.orchestra.run.smithi018.stdout:[client.2] 2023-03-24T22:24:37.078 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAHIx5k6t4JJBAArgi+AKRfXnFe5GC+Fjj9uA== 2023-03-24T22:24:37.078 INFO:teuthology.orchestra.run.smithi018.stdout:[client.1] 2023-03-24T22:24:37.078 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAHIx5kH+1rJxAAtwR+mTkYP9u0TcGEE/JvVQ== 2023-03-24T22:24:37.078 INFO:teuthology.orchestra.run.smithi018.stdout:[client.3] 2023-03-24T22:24:37.078 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQAHIx5kK+69KhAAXFbEQAaXZ3IiBO9hs2YmxQ== 2023-03-24T22:24:37.096 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2023-03-24T22:24:37.096 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAGIx5keIDFOhAA/bV/D8JE9IBHuz+eYIZgFA== 2023-03-24T22:24:37.097 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2023-03-24T22:24:37.097 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAHIx5kpcYXCBAA4sQYlV4RRHuORihYQIjR4A== 2023-03-24T22:24:37.097 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2023-03-24T22:24:37.097 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAHIx5kjSuUDxAAZvFferwYHJ1W5pVNVWqBbw== 2023-03-24T22:24:37.097 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2023-03-24T22:24:37.097 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAJIx5kgDkPARAAG7wjYfoBURGnp8+Gvs60YQ== 2023-03-24T22:24:37.097 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2023-03-24T22:24:37.098 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAMIx5kRqVKABAAz12j6eNrX0o/iBr/9cXHTQ== 2023-03-24T22:24:37.098 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2023-03-24T22:24:37.098 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAOIx5kDNDoOhAAVrzpJcQjEUAnQLepJ/BGBQ== 2023-03-24T22:24:37.098 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2023-03-24T22:24:37.098 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQARIx5kl8IiOhAAhNqbAoFbSCuY/+fmQ++wlQ== 2023-03-24T22:24:37.098 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2023-03-24T22:24:37.098 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAHIx5kyKeWBBAAfr+oDF6xm8MNNGwiAkA1RA== 2023-03-24T22:24:37.099 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2023-03-24T22:24:37.099 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAHIx5kXw+qFBAAkAs+SPbIoHheegrb+Do3gQ== 2023-03-24T22:24:37.099 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2023-03-24T22:24:37.099 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAHIx5kCusVHBAABLQhtx1pIytL9ukBSy/avA== 2023-03-24T22:24:37.099 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2023-03-24T22:24:37.099 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAYIx5kXJ15ChAAgJHDb9CgTB8yrinbU8H8Tw== 2023-03-24T22:24:37.099 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2023-03-24T22:24:37.099 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAbIx5kRF3dCBAA3RymeE8iQ48Uqpthph1VVw== 2023-03-24T22:24:37.100 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2023-03-24T22:24:37.100 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAeIx5kPzmlBxAAuBlbwACPmVOgjujWwYpPSg== 2023-03-24T22:24:37.100 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2023-03-24T22:24:37.100 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAhIx5k340KBhAA3z9NIrYnowFeDaoyO1c3ZA== 2023-03-24T22:24:37.100 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2023-03-24T22:24:37.100 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAHIx5k5AKTIBAAQQlDxeS1hC0KlxwRKUVpbA== 2023-03-24T22:24:37.100 INFO:teuthology.orchestra.run.smithi100.stdout:[client.2] 2023-03-24T22:24:37.101 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAHIx5k6t4JJBAArgi+AKRfXnFe5GC+Fjj9uA== 2023-03-24T22:24:37.101 INFO:teuthology.orchestra.run.smithi100.stdout:[client.1] 2023-03-24T22:24:37.101 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAHIx5kH+1rJxAAtwR+mTkYP9u0TcGEE/JvVQ== 2023-03-24T22:24:37.101 INFO:teuthology.orchestra.run.smithi100.stdout:[client.3] 2023-03-24T22:24:37.101 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAHIx5kK+69KhAAXFbEQAaXZ3IiBO9hs2YmxQ== 2023-03-24T22:24:37.102 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-24T22:24:37.122 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-24T22:24:37.195 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T22:24:37.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T22:24:37.254 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T22:24:37.256 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T22:24:37.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.368 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.370 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.425 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.427 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.481 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.483 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.538 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-24T22:24:37.540 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-24T22:24:37.594 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T22:24:37.596 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T22:24:37.653 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T22:24:37.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-24T22:24:37.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.714 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.768 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.770 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.824 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.826 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.880 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.882 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-24T22:24:37.936 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T22:24:37.938 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T22:24:37.995 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T22:24:37.997 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T22:24:38.054 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T22:24:38.056 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T22:24:38.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T22:24:38.115 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-24T22:24:38.172 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-03-24T22:24:38.172 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-03-24T22:24:38.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-24T22:24:38.300 DEBUG:teuthology.orchestra.run.smithi018:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-03-24T22:24:38.328 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-03-24T22:24:38.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-24T22:24:38.455 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-03-24T22:24:38.483 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-03-24T22:24:38.550 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-24T22:24:38.648 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-03-24T22:24:38.675 DEBUG:teuthology.orchestra.run.smithi018:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:38.680 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-24T22:24:38.732 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-03-24T22:24:38.733 INFO:tasks.ceph.mon.a:Restarting daemon 2023-03-24T22:24:38.734 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-03-24T22:24:38.738 INFO:tasks.ceph.mon.a:Started 2023-03-24T22:24:38.739 INFO:tasks.ceph.mon.b:Restarting daemon 2023-03-24T22:24:38.739 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-03-24T22:24:38.775 INFO:tasks.ceph.mon.b:Started 2023-03-24T22:24:38.775 INFO:tasks.ceph.mon.c:Restarting daemon 2023-03-24T22:24:38.777 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-03-24T22:24:38.779 INFO:tasks.ceph.mon.c:Started 2023-03-24T22:24:38.779 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-03-24T22:24:38.779 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-03-24T22:24:38.780 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-03-24T22:24:38.782 INFO:tasks.ceph.mgr.y:Started 2023-03-24T22:24:38.782 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-03-24T22:24:38.782 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-03-24T22:24:38.784 INFO:tasks.ceph.mgr.x:Started 2023-03-24T22:24:38.784 INFO:tasks.ceph:Setting crush tunables to default 2023-03-24T22:24:38.784 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd crush tunables default 2023-03-24T22:24:38.912 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.911+0000 7fd8c7947700 1 Processor -- start 2023-03-24T22:24:38.913 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.912+0000 7fd8c7947700 1 -- start start 2023-03-24T22:24:38.914 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c7947700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd8c015cc30 0x7fd8c015de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:38.914 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c7947700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd8c006bcc0 0x7fd8c00cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:38.915 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c7947700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c00cf2b0 con 0x7fd8c00cf740 2023-03-24T22:24:38.915 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c7947700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c00cfbe0 con 0x7fd8c006bcc0 2023-03-24T22:24:38.915 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c7947700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c00ce440 con 0x7fd8c015cc30 2023-03-24T22:24:38.915 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c5ee4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd8c006bcc0 0x7fd8c00cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:38.915 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c4ee2700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fd8c00cf740 0x7fd8c006b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:41050/0 (socket says 172.21.15.18:41050) 2023-03-24T22:24:38.915 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c5ee4700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fd8c006bcc0 0x7fd8c00cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56642/0 (socket says 172.21.15.18:56642) 2023-03-24T22:24:38.915 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c4ee2700 1 -- 172.21.15.18:0/3374014272 learned_addr learned my addr 172.21.15.18:0/3374014272 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:38.916 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c5ee4700 1 -- 172.21.15.18:0/3374014272 >> v2:172.21.15.18:3300/0 conn(0x7fd8c006bcc0 msgr2=0x7fd8c00cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-03-24T22:24:38.916 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c5ee4700 1 -- 172.21.15.18:0/3374014272 >> v2:172.21.15.18:3300/0 conn(0x7fd8c006bcc0 msgr2=0x7fd8c00cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-24T22:24:38.916 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c5ee4700 1 --2- 172.21.15.18:0/3374014272 >> v2:172.21.15.18:3300/0 conn(0x7fd8c006bcc0 0x7fd8c00cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-24T22:24:38.916 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c56e3700 1 --2- 172.21.15.18:0/3374014272 >> v2:172.21.15.100:3300/0 conn(0x7fd8c015cc30 0x7fd8c015de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:38.916 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c5ee4700 1 --2- 172.21.15.18:0/3374014272 >> v2:172.21.15.18:3300/0 conn(0x7fd8c006bcc0 0x7fd8c00cdf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-24T22:24:38.916 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c56e3700 1 -- 172.21.15.18:0/3374014272 >> v2:172.21.15.100:3300/0 conn(0x7fd8c015cc30 msgr2=0x7fd8c015de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-03-24T22:24:38.916 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c56e3700 1 -- 172.21.15.18:0/3374014272 >> v2:172.21.15.100:3300/0 conn(0x7fd8c015cc30 msgr2=0x7fd8c015de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-24T22:24:38.916 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c56e3700 1 --2- 172.21.15.18:0/3374014272 >> v2:172.21.15.100:3300/0 conn(0x7fd8c015cc30 0x7fd8c015de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-24T22:24:38.917 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:38.913+0000 7fd8c56e3700 1 --2- 172.21.15.18:0/3374014272 >> v2:172.21.15.100:3300/0 conn(0x7fd8c015cc30 0x7fd8c015de10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-24T22:24:39.101 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592952895 0 0) 0x7fd8c00cf2b0 con 0x7fd8c00cf740 2023-03-24T22:24:39.101 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8ac0057c0 con 0x7fd8c00cf740 2023-03-24T22:24:39.101 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 <== mon.0 v1:172.21.15.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3563225255 0 0) 0x7fd8ac0057c0 con 0x7fd8c00cf740 2023-03-24T22:24:39.101 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 >> v2:172.21.15.100:3300/0 conn(0x7fd8c015cc30 msgr2=0x7fd8c015de10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-24T22:24:39.102 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 --2- 172.21.15.18:0/3374014272 >> v2:172.21.15.100:3300/0 conn(0x7fd8c015cc30 0x7fd8c015de10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.102 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 >> v2:172.21.15.18:3300/0 conn(0x7fd8c006bcc0 msgr2=0x7fd8c00cdf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-24T22:24:39.102 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 --2- 172.21.15.18:0/3374014272 >> v2:172.21.15.18:3300/0 conn(0x7fd8c006bcc0 0x7fd8c00cdf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.102 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 --> v1:172.21.15.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00ce6a0 con 0x7fd8c00cf740 2023-03-24T22:24:39.102 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 <== mon.0 v1:172.21.15.18:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3658844337 0 0) 0x7fd8b0001670 con 0x7fd8c00cf740 2023-03-24T22:24:39.102 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 >> v1:172.21.15.18:6789/0 conn(0x7fd8c00cf740 legacy=0x7fd8c006b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:39.102 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8ac005c10 0x7fd8ac006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.103 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8ac006a60 0x7fd8ac006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.103 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8ac007410 0x7fd8ac00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.103 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00cfbe0 con 0x7fd8ac007410 2023-03-24T22:24:39.103 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00ce440 con 0x7fd8ac006a60 2023-03-24T22:24:39.103 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.098+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac00e2d0 con 0x7fd8ac005c10 2023-03-24T22:24:39.103 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8c4ee2700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8ac006a60 0x7fd8ac006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.103 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8c5ee4700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8ac007410 0x7fd8ac00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.103 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8c56e3700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8ac005c10 0x7fd8ac006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.104 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8c5ee4700 1 -- 172.21.15.18:0/3374014272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8ac005c10 msgr2=0x7fd8ac006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:39.104 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8c5ee4700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8ac005c10 0x7fd8ac006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.104 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8c5ee4700 1 -- 172.21.15.18:0/3374014272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8ac006a60 msgr2=0x7fd8ac006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:39.104 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8c5ee4700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8ac006a60 0x7fd8ac006ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.104 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8c5ee4700 1 -- 172.21.15.18:0/3374014272 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac0057c0 con 0x7fd8ac007410 2023-03-24T22:24:39.104 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8c5ee4700 1 -- 172.21.15.18:0/3374014272 reap_dead start 2023-03-24T22:24:39.104 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8c5ee4700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8ac007410 0x7fd8ac00dc20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd8c006c0b0 tx=0x7fd8ac007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34dd9c3aaf3c8b7 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:39.105 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8007df0 con 0x7fd8ac007410 2023-03-24T22:24:39.105 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b8003a20 con 0x7fd8ac007410 2023-03-24T22:24:39.105 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.099+0000 7fd8b7fff700 1 -- 172.21.15.18:0/3374014272 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8003dd0 con 0x7fd8ac007410 2023-03-24T22:24:39.105 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.100+0000 7fd8c7947700 1 -- 172.21.15.18:0/3374014272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8ac007410 msgr2=0x7fd8ac00dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:39.105 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.100+0000 7fd8c7947700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8ac007410 0x7fd8ac00dc20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd8c006c0b0 tx=0x7fd8ac007a90 comp rx=0 tx=0).stop 2023-03-24T22:24:39.105 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.100+0000 7fd8c7947700 1 -- 172.21.15.18:0/3374014272 shutdown_connections 2023-03-24T22:24:39.105 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.100+0000 7fd8c7947700 1 --2- 172.21.15.18:0/3374014272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8ac007410 0x7fd8ac00dc20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.105 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.100+0000 7fd8c7947700 1 -- 172.21.15.18:0/3374014272 >> 172.21.15.18:0/3374014272 conn(0x7fd8c00d0a00 msgr2=0x7fd8c00cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:39.106 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.100+0000 7fd8c7947700 1 -- 172.21.15.18:0/3374014272 shutdown_connections 2023-03-24T22:24:39.106 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.100+0000 7fd8c7947700 1 -- 172.21.15.18:0/3374014272 wait complete. 2023-03-24T22:24:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.101+0000 7fd8c7947700 1 Processor -- start 2023-03-24T22:24:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.101+0000 7fd8c7947700 1 -- start start 2023-03-24T22:24:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c7947700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8ac007410 0x7fd8c0154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c7947700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c00adc50 0x7fd8c00ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c7947700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c0155690 0x7fd8c0155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c7947700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00acbb0 con 0x7fd8c00adc50 2023-03-24T22:24:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c7947700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00ace90 con 0x7fd8ac007410 2023-03-24T22:24:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c7947700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00ad170 con 0x7fd8c0155690 2023-03-24T22:24:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c4ee2700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c00adc50 0x7fd8c00ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c56e3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8ac007410 0x7fd8c0154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c4ee2700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c00adc50 0x7fd8c00ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56666/0 (socket says 172.21.15.18:56666) 2023-03-24T22:24:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c4ee2700 1 -- 172.21.15.18:0/3760502948 learned_addr learned my addr 172.21.15.18:0/3760502948 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.102+0000 7fd8c5ee4700 1 --2- 172.21.15.18:0/3760502948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c0155690 0x7fd8c0155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8c4ee2700 1 -- 172.21.15.18:0/3760502948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c0155690 msgr2=0x7fd8c0155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:39.110 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8c4ee2700 1 --2- 172.21.15.18:0/3760502948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c0155690 0x7fd8c0155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.110 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8c4ee2700 1 -- 172.21.15.18:0/3760502948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8ac007410 msgr2=0x7fd8c0154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:39.110 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8c4ee2700 1 --2- 172.21.15.18:0/3760502948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8ac007410 0x7fd8c0154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.110 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8c4ee2700 1 -- 172.21.15.18:0/3760502948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00cec60 con 0x7fd8c00adc50 2023-03-24T22:24:39.110 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8c4ee2700 1 --2- 172.21.15.18:0/3760502948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c00adc50 0x7fd8c00ae0c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0007310 tx=0x7fd8b0007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:39.110 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8b5ffb700 1 -- 172.21.15.18:0/3760502948 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0011580 con 0x7fd8c00adc50 2023-03-24T22:24:39.110 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8b5ffb700 1 -- 172.21.15.18:0/3760502948 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b0011720 con 0x7fd8c00adc50 2023-03-24T22:24:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8b5ffb700 1 -- 172.21.15.18:0/3760502948 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0011ad0 con 0x7fd8c00adc50 2023-03-24T22:24:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8c7947700 1 -- 172.21.15.18:0/3760502948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c00ce980 con 0x7fd8c00adc50 2023-03-24T22:24:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8b5ffb700 1 -- 172.21.15.18:0/3760502948 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd8b001d460 con 0x7fd8c00adc50 2023-03-24T22:24:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8c7947700 1 -- 172.21.15.18:0/3760502948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c00cef40 con 0x7fd8c00adc50 2023-03-24T22:24:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.103+0000 7fd8b5ffb700 1 -- 172.21.15.18:0/3760502948 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd8b001d960 con 0x7fd8c00adc50 2023-03-24T22:24:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.104+0000 7fd8c7947700 1 -- 172.21.15.18:0/3760502948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c00b5d40 con 0x7fd8c00adc50 2023-03-24T22:24:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.107+0000 7fd8b5ffb700 1 -- 172.21.15.18:0/3760502948 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fd8b001dbd0 con 0x7fd8c00adc50 2023-03-24T22:24:39.187 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.185+0000 7fd8c7947700 1 -- 172.21.15.18:0/3760502948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd8c00cd5b0 con 0x7fd8c00adc50 2023-03-24T22:24:39.188 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.187+0000 7fd8b5ffb700 1 -- 172.21.15.18:0/3760502948 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd8b001dd70 con 0x7fd8c00adc50 2023-03-24T22:24:39.190 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.188+0000 7fd8c7947700 1 -- 172.21.15.18:0/3760502948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c00adc50 msgr2=0x7fd8c00ae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:39.190 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.188+0000 7fd8c7947700 1 --2- 172.21.15.18:0/3760502948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c00adc50 0x7fd8c00ae0c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0007310 tx=0x7fd8b0007620 comp rx=0 tx=0).stop 2023-03-24T22:24:39.190 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.188+0000 7fd8c7947700 1 -- 172.21.15.18:0/3760502948 shutdown_connections 2023-03-24T22:24:39.190 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.188+0000 7fd8c7947700 1 --2- 172.21.15.18:0/3760502948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c0155690 0x7fd8c0155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.190 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.188+0000 7fd8c7947700 1 --2- 172.21.15.18:0/3760502948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8ac007410 0x7fd8c0154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.190 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.188+0000 7fd8c7947700 1 --2- 172.21.15.18:0/3760502948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8c00adc50 0x7fd8c00ae0c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.191 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.188+0000 7fd8c7947700 1 -- 172.21.15.18:0/3760502948 >> 172.21.15.18:0/3760502948 conn(0x7fd8c00d0a00 msgr2=0x7fd8c015bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:39.191 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.188+0000 7fd8c7947700 1 -- 172.21.15.18:0/3760502948 shutdown_connections 2023-03-24T22:24:39.191 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:39.188+0000 7fd8c7947700 1 -- 172.21.15.18:0/3760502948 wait complete. 2023-03-24T22:24:39.191 INFO:teuthology.orchestra.run.smithi018.stderr:adjusted tunables profile to default 2023-03-24T22:24:39.201 INFO:tasks.ceph:check_enable_crimson: False 2023-03-24T22:24:39.201 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-03-24T22:24:39.202 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:39.202 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-03-24T22:24:39.229 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:39.230 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-03-24T22:24:39.299 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:39.300 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-03-24T22:24:39.368 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2023-03-24T22:24:39.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-03-24T22:24:39.436 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:39.437 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-03-24T22:24:39.464 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:39.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-03-24T22:24:39.531 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:39.532 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-03-24T22:24:39.599 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-03-24T22:24:39.599 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-03-24T22:24:39.666 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new d184f8d0-aaf7-421c-9c33-c9f32a0e9236 0 2023-03-24T22:24:39.829 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.827+0000 7f979a922700 1 Processor -- start 2023-03-24T22:24:39.829 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.828+0000 7f979a922700 1 -- start start 2023-03-24T22:24:39.831 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f979a922700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f97940b56e0 0x7f97940cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.831 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f979a922700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f979406abb0 0x7f979406bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.831 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f979a922700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97940712a0 con 0x7f979406b7e0 2023-03-24T22:24:39.831 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f979a922700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97940b5ab0 con 0x7f97940b56e0 2023-03-24T22:24:39.831 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f979a922700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979406c190 con 0x7f979406abb0 2023-03-24T22:24:39.831 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f9798ebf700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f979406abb0 0x7f979406bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.832 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f9798ebf700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f979406abb0 0x7f979406bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40840/0 (socket says 172.21.15.100:40840) 2023-03-24T22:24:39.832 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f9798ebf700 1 -- 172.21.15.100:0/405200538 learned_addr learned my addr 172.21.15.100:0/405200538 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:39.832 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f97937fe700 1 --1- 172.21.15.100:0/405200538 >> v1:172.21.15.100:6789/0 conn(0x7f979406b7e0 0x7f979406a4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46646/0 (socket says 172.21.15.100:46646) 2023-03-24T22:24:39.832 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f9793fff700 1 --2- 172.21.15.100:0/405200538 >> v2:172.21.15.18:3300/0 conn(0x7f97940b56e0 0x7f97940cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.832 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f9798ebf700 1 -- 172.21.15.100:0/405200538 >> v2:172.21.15.18:3300/0 conn(0x7f97940b56e0 msgr2=0x7f97940cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:39.832 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f9798ebf700 1 --2- 172.21.15.100:0/405200538 >> v2:172.21.15.18:3300/0 conn(0x7f97940b56e0 0x7f97940cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.832 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f9798ebf700 1 -- 172.21.15.100:0/405200538 >> v1:172.21.15.100:6789/0 conn(0x7f979406b7e0 legacy=0x7f979406a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:39.833 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f9798ebf700 1 -- 172.21.15.100:0/405200538 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940cc6b0 con 0x7f979406abb0 2023-03-24T22:24:39.833 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f9792ffd700 1 -- 172.21.15.100:0/405200538 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218710954 0 0) 0x7f97940712a0 con 0x7f979406b7e0 2023-03-24T22:24:39.833 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.829+0000 7f9798ebf700 1 --2- 172.21.15.100:0/405200538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 0x7f979406bc50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f979415adf0 tx=0x7f978c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9935024e69ad4d4 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:39.833 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f9792ffd700 1 -- 172.21.15.100:0/405200538 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c00e070 con 0x7f979406abb0 2023-03-24T22:24:39.833 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f9792ffd700 1 -- 172.21.15.100:0/405200538 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c00b910 con 0x7f979406abb0 2023-03-24T22:24:39.833 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f9792ffd700 1 -- 172.21.15.100:0/405200538 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c00bcc0 con 0x7f979406abb0 2023-03-24T22:24:39.836 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f979a922700 1 -- 172.21.15.100:0/405200538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 msgr2=0x7f979406bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:39.836 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f979a922700 1 --2- 172.21.15.100:0/405200538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 0x7f979406bc50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f979415adf0 tx=0x7f978c00b330 comp rx=0 tx=0).stop 2023-03-24T22:24:39.836 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f979a922700 1 -- 172.21.15.100:0/405200538 shutdown_connections 2023-03-24T22:24:39.836 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f979a922700 1 --2- 172.21.15.100:0/405200538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 0x7f979406bc50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.836 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f979a922700 1 --2- 172.21.15.100:0/405200538 >> v2:172.21.15.18:3300/0 conn(0x7f97940b56e0 0x7f97940cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.836 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f979a922700 1 -- 172.21.15.100:0/405200538 >> 172.21.15.100:0/405200538 conn(0x7f97940d0a00 msgr2=0x7f97940cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:39.836 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f979a922700 1 -- 172.21.15.100:0/405200538 shutdown_connections 2023-03-24T22:24:39.837 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.830+0000 7f979a922700 1 -- 172.21.15.100:0/405200538 wait complete. 2023-03-24T22:24:39.837 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.831+0000 7f979a922700 1 Processor -- start 2023-03-24T22:24:39.837 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.832+0000 7f979a922700 1 -- start start 2023-03-24T22:24:39.837 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.832+0000 7f979a922700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 0x7f97941ef690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.837 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.832+0000 7f979a922700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f979406b7e0 0x7f97941efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.837 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.832+0000 7f979a922700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97940b56e0 0x7f97940c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:39.838 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.832+0000 7f979a922700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97940c3e10 con 0x7f979406b7e0 2023-03-24T22:24:39.838 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.832+0000 7f979a922700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97941f0110 con 0x7f979406abb0 2023-03-24T22:24:39.838 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.832+0000 7f979a922700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97940c46b0 con 0x7f97940b56e0 2023-03-24T22:24:39.838 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9793fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 0x7f97941ef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.838 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9798ebf700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97940b56e0 0x7f97940c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.838 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9793fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 0x7f97941ef690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40842/0 (socket says 172.21.15.100:40842) 2023-03-24T22:24:39.838 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9798ebf700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97940b56e0 0x7f97940c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:34082/0 (socket says 172.21.15.100:34082) 2023-03-24T22:24:39.838 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9793fff700 1 -- 172.21.15.100:0/1101739833 learned_addr learned my addr 172.21.15.100:0/1101739833 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:39.839 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f97937fe700 1 --2- 172.21.15.100:0/1101739833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f979406b7e0 0x7f97941efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:39.839 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9798ebf700 1 -- 172.21.15.100:0/1101739833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 msgr2=0x7f97941ef690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:39.839 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9798ebf700 1 --2- 172.21.15.100:0/1101739833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 0x7f97941ef690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.839 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9798ebf700 1 -- 172.21.15.100:0/1101739833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f979406b7e0 msgr2=0x7f97941efbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:39.839 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9798ebf700 1 --2- 172.21.15.100:0/1101739833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f979406b7e0 0x7f97941efbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.839 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9798ebf700 1 -- 172.21.15.100:0/1101739833 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c009b30 con 0x7f97940b56e0 2023-03-24T22:24:39.839 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9793fff700 1 --2- 172.21.15.100:0/1101739833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 0x7f97941ef690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:39.840 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9798ebf700 1 --2- 172.21.15.100:0/1101739833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97940b56e0 0x7f97940c5ea0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f97940cd240 tx=0x7f978c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:39.840 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9790ff9700 1 -- 172.21.15.100:0/1101739833 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c004900 con 0x7f97940b56e0 2023-03-24T22:24:39.840 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9790ff9700 1 -- 172.21.15.100:0/1101739833 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c0092e0 con 0x7f97940b56e0 2023-03-24T22:24:39.840 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f979a922700 1 -- 172.21.15.100:0/1101739833 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979406b360 con 0x7f97940b56e0 2023-03-24T22:24:39.840 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9790ff9700 1 -- 172.21.15.100:0/1101739833 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c01c680 con 0x7f97940b56e0 2023-03-24T22:24:39.840 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f979a922700 1 -- 172.21.15.100:0/1101739833 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979406b080 con 0x7f97940b56e0 2023-03-24T22:24:39.840 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9790ff9700 1 -- 172.21.15.100:0/1101739833 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f978c01cae0 con 0x7f97940b56e0 2023-03-24T22:24:39.840 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.833+0000 7f9790ff9700 1 -- 172.21.15.100:0/1101739833 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f978c025c90 con 0x7f97940b56e0 2023-03-24T22:24:39.841 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.834+0000 7f979a922700 1 -- 172.21.15.100:0/1101739833 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97940cb690 con 0x7f97940b56e0 2023-03-24T22:24:39.841 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.836+0000 7f9790ff9700 1 -- 172.21.15.100:0/1101739833 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f978c020070 con 0x7f97940b56e0 2023-03-24T22:24:39.921 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.920+0000 7f979a922700 1 -- 172.21.15.100:0/1101739833 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d184f8d0-aaf7-421c-9c33-c9f32a0e9236", "id": 0} v 0) v1 -- 0x7f979404e850 con 0x7f97940b56e0 2023-03-24T22:24:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.923+0000 7f9790ff9700 1 -- 172.21.15.100:0/1101739833 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d184f8d0-aaf7-421c-9c33-c9f32a0e9236", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f978c025450 con 0x7f97940b56e0 2023-03-24T22:24:39.924 INFO:teuthology.orchestra.run.smithi100.stdout:0 2023-03-24T22:24:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.924+0000 7f979a922700 1 -- 172.21.15.100:0/1101739833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97940b56e0 msgr2=0x7f97940c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.924+0000 7f979a922700 1 --2- 172.21.15.100:0/1101739833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97940b56e0 0x7f97940c5ea0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f97940cd240 tx=0x7f978c004ad0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.924+0000 7f979a922700 1 -- 172.21.15.100:0/1101739833 shutdown_connections 2023-03-24T22:24:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.924+0000 7f979a922700 1 --2- 172.21.15.100:0/1101739833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97940b56e0 0x7f97940c5ea0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.924+0000 7f979a922700 1 --2- 172.21.15.100:0/1101739833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979406abb0 0x7f97941ef690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.924+0000 7f979a922700 1 --2- 172.21.15.100:0/1101739833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f979406b7e0 0x7f97941efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.924+0000 7f979a922700 1 -- 172.21.15.100:0/1101739833 >> 172.21.15.100:0/1101739833 conn(0x7f97940d0a00 msgr2=0x7f979415b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.924+0000 7f979a922700 1 -- 172.21.15.100:0/1101739833 shutdown_connections 2023-03-24T22:24:39.926 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:39.924+0000 7f979a922700 1 -- 172.21.15.100:0/1101739833 wait complete. 2023-03-24T22:24:39.938 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new d46c78b3-52ad-41f3-8c64-3d37e9adee63 1 2023-03-24T22:24:40.057 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:40.056+0000 7f48f8982200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-24T22:24:40.060 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.059+0000 7ff813a38700 1 Processor -- start 2023-03-24T22:24:40.060 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.059+0000 7ff813a38700 1 -- start start 2023-03-24T22:24:40.061 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff813a38700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c0afe70 0x7ff80c0ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.061 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff813a38700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c0b0460 con 0x7ff80c15a980 2023-03-24T22:24:40.062 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff813a38700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c0b0740 con 0x7ff80c0cf250 2023-03-24T22:24:40.062 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff813a38700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c15ad50 con 0x7ff80c0afe70 2023-03-24T22:24:40.062 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff810fd3700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff80c0cf250 0x7ff80c06ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46662/0 (socket says 172.21.15.100:46662) 2023-03-24T22:24:40.062 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff811fd5700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c0afe70 0x7ff80c0ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.062 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff810fd3700 1 -- 172.21.15.100:0/2501120629 learned_addr learned my addr 172.21.15.100:0/2501120629 (peer_addr_for_me v1:172.21.15.100:0/0) 2023-03-24T22:24:40.062 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff811fd5700 1 -- 172.21.15.100:0/2501120629 >> v1:172.21.15.100:6789/0 conn(0x7ff80c0cf250 legacy=0x7ff80c06ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.062 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff811fd5700 1 -- 172.21.15.100:0/2501120629 >> v1:172.21.15.18:6789/0 conn(0x7ff80c15a980 legacy=0x7ff80c0ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.063 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff811fd5700 1 -- 172.21.15.100:0/2501120629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c06df20 con 0x7ff80c0afe70 2023-03-24T22:24:40.063 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff803fff700 1 -- 172.21.15.100:0/2501120629 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330919510 0 0) 0x7ff80c0b0740 con 0x7ff80c0cf250 2023-03-24T22:24:40.063 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff811fd5700 1 --2- 172.21.15.100:0/2501120629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c0afe70 0x7ff80c0ce360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff804009dc0 tx=0x7ff80400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af655cbd91e9b77 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:40.063 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff803fff700 1 -- 172.21.15.100:0/2501120629 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80400e070 con 0x7ff80c0afe70 2023-03-24T22:24:40.063 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff803fff700 1 -- 172.21.15.100:0/2501120629 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80400bbd0 con 0x7ff80c0afe70 2023-03-24T22:24:40.063 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.060+0000 7ff803fff700 1 -- 172.21.15.100:0/2501120629 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804012430 con 0x7ff80c0afe70 2023-03-24T22:24:40.065 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.061+0000 7ff813a38700 1 -- 172.21.15.100:0/2501120629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c0afe70 msgr2=0x7ff80c0ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.066 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.061+0000 7ff813a38700 1 --2- 172.21.15.100:0/2501120629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c0afe70 0x7ff80c0ce360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff804009dc0 tx=0x7ff80400b330 comp rx=0 tx=0).stop 2023-03-24T22:24:40.066 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.061+0000 7ff813a38700 1 -- 172.21.15.100:0/2501120629 shutdown_connections 2023-03-24T22:24:40.066 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.061+0000 7ff813a38700 1 --2- 172.21.15.100:0/2501120629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c0afe70 0x7ff80c0ce360 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.066 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.061+0000 7ff813a38700 1 -- 172.21.15.100:0/2501120629 >> 172.21.15.100:0/2501120629 conn(0x7ff80c09a290 msgr2=0x7ff80c0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:40.066 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.061+0000 7ff813a38700 1 -- 172.21.15.100:0/2501120629 shutdown_connections 2023-03-24T22:24:40.066 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.061+0000 7ff813a38700 1 -- 172.21.15.100:0/2501120629 wait complete. 2023-03-24T22:24:40.067 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.062+0000 7ff813a38700 1 Processor -- start 2023-03-24T22:24:40.067 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.063+0000 7ff813a38700 1 -- start start 2023-03-24T22:24:40.067 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.063+0000 7ff813a38700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff80c0afe70 0x7ff80c146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.067 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.063+0000 7ff813a38700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c0cf250 0x7ff80c0c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.067 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff813a38700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c15a980 0x7ff80c0b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.067 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff813a38700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c0b57c0 con 0x7ff80c0afe70 2023-03-24T22:24:40.067 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff813a38700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c1468a0 con 0x7ff80c0cf250 2023-03-24T22:24:40.067 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff813a38700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c06fc40 con 0x7ff80c15a980 2023-03-24T22:24:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff811fd5700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c15a980 0x7ff80c0b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff810fd3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c0cf250 0x7ff80c0c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff810fd3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c0cf250 0x7ff80c0c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40854/0 (socket says 172.21.15.100:40854) 2023-03-24T22:24:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff811fd5700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c15a980 0x7ff80c0b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:34092/0 (socket says 172.21.15.100:34092) 2023-03-24T22:24:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff810fd3700 1 -- 172.21.15.100:0/3129685123 learned_addr learned my addr 172.21.15.100:0/3129685123 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff8117d4700 1 --2- 172.21.15.100:0/3129685123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff80c0afe70 0x7ff80c146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.068 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff811fd5700 1 -- 172.21.15.100:0/3129685123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c0cf250 msgr2=0x7ff80c0c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff811fd5700 1 --2- 172.21.15.100:0/3129685123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c0cf250 0x7ff80c0c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff811fd5700 1 -- 172.21.15.100:0/3129685123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff80c0afe70 msgr2=0x7ff80c146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff811fd5700 1 --2- 172.21.15.100:0/3129685123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff80c0afe70 0x7ff80c146360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff811fd5700 1 -- 172.21.15.100:0/3129685123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804009a70 con 0x7ff80c15a980 2023-03-24T22:24:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff810fd3700 1 --2- 172.21.15.100:0/3129685123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c0cf250 0x7ff80c0c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff811fd5700 1 --2- 172.21.15.100:0/3129685123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c15a980 0x7ff80c0b64f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff804016000 tx=0x7ff804004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:40.069 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff801ffb700 1 -- 172.21.15.100:0/3129685123 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80401b810 con 0x7ff80c15a980 2023-03-24T22:24:40.070 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff801ffb700 1 -- 172.21.15.100:0/3129685123 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff804020070 con 0x7ff80c15a980 2023-03-24T22:24:40.070 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff813a38700 1 -- 172.21.15.100:0/3129685123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c06e4e0 con 0x7ff80c15a980 2023-03-24T22:24:40.070 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff801ffb700 1 -- 172.21.15.100:0/3129685123 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80401bbd0 con 0x7ff80c15a980 2023-03-24T22:24:40.070 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.064+0000 7ff813a38700 1 -- 172.21.15.100:0/3129685123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c06e200 con 0x7ff80c15a980 2023-03-24T22:24:40.070 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.065+0000 7ff801ffb700 1 -- 172.21.15.100:0/3129685123 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff804027780 con 0x7ff80c15a980 2023-03-24T22:24:40.070 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.065+0000 7ff801ffb700 1 -- 172.21.15.100:0/3129685123 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff804022070 con 0x7ff80c15a980 2023-03-24T22:24:40.071 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.065+0000 7ff813a38700 1 -- 172.21.15.100:0/3129685123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80c0d1d70 con 0x7ff80c15a980 2023-03-24T22:24:40.071 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.068+0000 7ff801ffb700 1 -- 172.21.15.100:0/3129685123 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7ff80402c060 con 0x7ff80c15a980 2023-03-24T22:24:40.083 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:40.082+0000 7f1e5e1b4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-24T22:24:40.132 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:40.131+0000 7f48f8982200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-24T22:24:40.153 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.151+0000 7ff813a38700 1 -- 172.21.15.100:0/3129685123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d46c78b3-52ad-41f3-8c64-3d37e9adee63", "id": 1} v 0) v1 -- 0x7ff80c04e850 con 0x7ff80c15a980 2023-03-24T22:24:40.155 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.154+0000 7ff801ffb700 1 -- 172.21.15.100:0/3129685123 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d46c78b3-52ad-41f3-8c64-3d37e9adee63", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff804027bc0 con 0x7ff80c15a980 2023-03-24T22:24:40.156 INFO:teuthology.orchestra.run.smithi100.stdout:1 2023-03-24T22:24:40.157 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.155+0000 7ff813a38700 1 -- 172.21.15.100:0/3129685123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c15a980 msgr2=0x7ff80c0b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.157 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.155+0000 7ff813a38700 1 --2- 172.21.15.100:0/3129685123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c15a980 0x7ff80c0b64f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff804016000 tx=0x7ff804004ad0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.157 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.155+0000 7ff813a38700 1 -- 172.21.15.100:0/3129685123 shutdown_connections 2023-03-24T22:24:40.157 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.155+0000 7ff813a38700 1 --2- 172.21.15.100:0/3129685123 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff80c0afe70 0x7ff80c146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.157 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.155+0000 7ff813a38700 1 --2- 172.21.15.100:0/3129685123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff80c15a980 0x7ff80c0b64f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.157 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.155+0000 7ff813a38700 1 --2- 172.21.15.100:0/3129685123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff80c0cf250 0x7ff80c0c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.156+0000 7ff813a38700 1 -- 172.21.15.100:0/3129685123 >> 172.21.15.100:0/3129685123 conn(0x7ff80c09a290 msgr2=0x7ff80c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.156+0000 7ff813a38700 1 -- 172.21.15.100:0/3129685123 shutdown_connections 2023-03-24T22:24:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.156+0000 7ff813a38700 1 -- 172.21.15.100:0/3129685123 wait complete. 2023-03-24T22:24:40.158 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:40.156+0000 7f1e5e1b4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-24T22:24:40.168 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 4e4e783f-f720-42f9-b394-002a79f5eb0c 2 2023-03-24T22:24:40.222 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:40.221+0000 7f48f8982200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-24T22:24:40.255 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:40.254+0000 7f1e5e1b4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-24T22:24:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.290+0000 7fcf60fb3700 1 Processor -- start 2023-03-24T22:24:40.291 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.290+0000 7fcf60fb3700 1 -- start start 2023-03-24T22:24:40.293 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.290+0000 7fcf60fb3700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcf5c0b23c0 0x7fcf5c0b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.293 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.290+0000 7fcf60fb3700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fcf5c0d01e0 0x7fcf5c0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.293 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.290+0000 7fcf60fb3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5c0d2830 con 0x7fcf5c0d1710 2023-03-24T22:24:40.293 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.290+0000 7fcf60fb3700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5c0b2790 con 0x7fcf5c0d01e0 2023-03-24T22:24:40.293 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.290+0000 7fcf60fb3700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5c0d2df0 con 0x7fcf5c0b23c0 2023-03-24T22:24:40.293 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf5a59c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcf5c0b23c0 0x7fcf5c0b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.293 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf5a59c700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcf5c0b23c0 0x7fcf5c0b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40862/0 (socket says 172.21.15.100:40862) 2023-03-24T22:24:40.294 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf5a59c700 1 -- 172.21.15.100:0/1542428925 learned_addr learned my addr 172.21.15.100:0/1542428925 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:40.294 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf5ad9d700 1 --2- 172.21.15.100:0/1542428925 >> v2:172.21.15.18:3300/0 conn(0x7fcf5c0d01e0 0x7fcf5c0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.294 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf5a59c700 1 -- 172.21.15.100:0/1542428925 >> v2:172.21.15.18:3300/0 conn(0x7fcf5c0d01e0 msgr2=0x7fcf5c0d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:40.294 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf5a59c700 1 --2- 172.21.15.100:0/1542428925 >> v2:172.21.15.18:3300/0 conn(0x7fcf5c0d01e0 0x7fcf5c0d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.294 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf5a59c700 1 -- 172.21.15.100:0/1542428925 >> v1:172.21.15.18:6789/0 conn(0x7fcf5c0d1710 legacy=0x7fcf5c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.294 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf5a59c700 1 -- 172.21.15.100:0/1542428925 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c0cd920 con 0x7fcf5c0b23c0 2023-03-24T22:24:40.294 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf5a59c700 1 --2- 172.21.15.100:0/1542428925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf5c0b23c0 0x7fcf5c0b5f80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c067810 tx=0x7fcf4c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4625beeb97d4a6e server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:40.295 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf59d9b700 1 -- 172.21.15.100:0/1542428925 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c011830 con 0x7fcf5c0b23c0 2023-03-24T22:24:40.295 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf59d9b700 1 -- 172.21.15.100:0/1542428925 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c011990 con 0x7fcf5c0b23c0 2023-03-24T22:24:40.295 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.291+0000 7fcf59d9b700 1 -- 172.21.15.100:0/1542428925 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c011d20 con 0x7fcf5c0b23c0 2023-03-24T22:24:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.292+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1542428925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf5c0b23c0 msgr2=0x7fcf5c0b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.292+0000 7fcf60fb3700 1 --2- 172.21.15.100:0/1542428925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf5c0b23c0 0x7fcf5c0b5f80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c067810 tx=0x7fcf4c009bd0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.297 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.292+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1542428925 shutdown_connections 2023-03-24T22:24:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.292+0000 7fcf60fb3700 1 --2- 172.21.15.100:0/1542428925 >> v2:172.21.15.18:3300/0 conn(0x7fcf5c0d01e0 0x7fcf5c0d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.292+0000 7fcf60fb3700 1 --2- 172.21.15.100:0/1542428925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf5c0b23c0 0x7fcf5c0b5f80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.292+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1542428925 >> 172.21.15.100:0/1542428925 conn(0x7fcf5c066140 msgr2=0x7fcf5c066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.292+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1542428925 shutdown_connections 2023-03-24T22:24:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.292+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1542428925 wait complete. 2023-03-24T22:24:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.293+0000 7fcf60fb3700 1 Processor -- start 2023-03-24T22:24:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.293+0000 7fcf60fb3700 1 -- start start 2023-03-24T22:24:40.298 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf60fb3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf5c0b23c0 0x7fcf5c15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf60fb3700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf5c0d01e0 0x7fcf5c0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf60fb3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf5c0d1710 0x7fcf5c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf60fb3700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0c99a0 con 0x7fcf5c0d1710 2023-03-24T22:24:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf60fb3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0c83c0 con 0x7fcf5c0b23c0 2023-03-24T22:24:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf60fb3700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0c86a0 con 0x7fcf5c0d01e0 2023-03-24T22:24:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf53fff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf5c0d01e0 0x7fcf5c0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.299 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf5a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf5c0b23c0 0x7fcf5c15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf53fff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf5c0d01e0 0x7fcf5c0c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:34098/0 (socket says 172.21.15.100:34098) 2023-03-24T22:24:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf53fff700 1 -- 172.21.15.100:0/1175750083 learned_addr learned my addr 172.21.15.100:0/1175750083 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.294+0000 7fcf5ad9d700 1 --2- 172.21.15.100:0/1175750083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf5c0d1710 0x7fcf5c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf5ad9d700 1 -- 172.21.15.100:0/1175750083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf5c0d01e0 msgr2=0x7fcf5c0c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf5ad9d700 1 --2- 172.21.15.100:0/1175750083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf5c0d01e0 0x7fcf5c0c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf5ad9d700 1 -- 172.21.15.100:0/1175750083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf5c0b23c0 msgr2=0x7fcf5c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf5ad9d700 1 --2- 172.21.15.100:0/1175750083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf5c0b23c0 0x7fcf5c15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.300 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf5ad9d700 1 -- 172.21.15.100:0/1175750083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c009550 con 0x7fcf5c0d1710 2023-03-24T22:24:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf5a59c700 1 --2- 172.21.15.100:0/1175750083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf5c0b23c0 0x7fcf5c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf53fff700 1 --2- 172.21.15.100:0/1175750083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf5c0d01e0 0x7fcf5c0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf5ad9d700 1 --2- 172.21.15.100:0/1175750083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf5c0d1710 0x7fcf5c0cbb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400eba0 tx=0x7fcf5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf52ffd700 1 -- 172.21.15.100:0/1175750083 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54009a80 con 0x7fcf5c0d1710 2023-03-24T22:24:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf52ffd700 1 -- 172.21.15.100:0/1175750083 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5400f040 con 0x7fcf5c0d1710 2023-03-24T22:24:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1175750083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c0d2a80 con 0x7fcf5c0d1710 2023-03-24T22:24:40.301 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf52ffd700 1 -- 172.21.15.100:0/1175750083 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400cc40 con 0x7fcf5c0d1710 2023-03-24T22:24:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1175750083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c0cdc00 con 0x7fcf5c0d1710 2023-03-24T22:24:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf52ffd700 1 -- 172.21.15.100:0/1175750083 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcf54014780 con 0x7fcf5c0d1710 2023-03-24T22:24:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.295+0000 7fcf52ffd700 1 -- 172.21.15.100:0/1175750083 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fcf5400cda0 con 0x7fcf5c0d1710 2023-03-24T22:24:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.296+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1175750083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5c0cf530 con 0x7fcf5c0d1710 2023-03-24T22:24:40.302 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.298+0000 7fcf52ffd700 1 -- 172.21.15.100:0/1175750083 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fcf5403bb10 con 0x7fcf5c0d1710 2023-03-24T22:24:40.382 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.381+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1175750083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4e4e783f-f720-42f9-b394-002a79f5eb0c", "id": 2} v 0) v1 -- 0x7fcf5c04e850 con 0x7fcf5c0d1710 2023-03-24T22:24:40.385 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.384+0000 7fcf52ffd700 1 -- 172.21.15.100:0/1175750083 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4e4e783f-f720-42f9-b394-002a79f5eb0c", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcf5402a070 con 0x7fcf5c0d1710 2023-03-24T22:24:40.385 INFO:teuthology.orchestra.run.smithi100.stdout:2 2023-03-24T22:24:40.386 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.385+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1175750083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf5c0d1710 msgr2=0x7fcf5c0cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.386 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.385+0000 7fcf60fb3700 1 --2- 172.21.15.100:0/1175750083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf5c0d1710 0x7fcf5c0cbb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400eba0 tx=0x7fcf5400eeb0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.386 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.385+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1175750083 shutdown_connections 2023-03-24T22:24:40.386 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.385+0000 7fcf60fb3700 1 --2- 172.21.15.100:0/1175750083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf5c0d1710 0x7fcf5c0cbb80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.387 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.385+0000 7fcf60fb3700 1 --2- 172.21.15.100:0/1175750083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf5c0b23c0 0x7fcf5c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.387 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.385+0000 7fcf60fb3700 1 --2- 172.21.15.100:0/1175750083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf5c0d01e0 0x7fcf5c0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.387 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.385+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1175750083 >> 172.21.15.100:0/1175750083 conn(0x7fcf5c066140 msgr2=0x7fcf5c15bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:40.387 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.385+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1175750083 shutdown_connections 2023-03-24T22:24:40.387 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.385+0000 7fcf60fb3700 1 -- 172.21.15.100:0/1175750083 wait complete. 2023-03-24T22:24:40.397 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 2a8e5486-0ae6-43c3-870e-07ad88a43340 3 2023-03-24T22:24:40.417 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:40.417+0000 7f48f8982200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-24T22:24:40.451 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:40.450+0000 7f1e5e1b4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-24T22:24:40.519 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.517+0000 7ffa180eb700 1 Processor -- start 2023-03-24T22:24:40.519 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.517+0000 7ffa180eb700 1 -- start start 2023-03-24T22:24:40.520 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.518+0000 7ffa180eb700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7ffa101610d0 0x7ffa100aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.520 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.518+0000 7ffa180eb700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa101607c0 con 0x7ffa101614a0 2023-03-24T22:24:40.521 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.518+0000 7ffa180eb700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1015d0a0 con 0x7ffa10071680 2023-03-24T22:24:40.521 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.518+0000 7ffa180eb700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa100af500 con 0x7ffa101610d0 2023-03-24T22:24:40.521 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.518+0000 7ffa16688700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ffa10071680 0x7ffa1006a4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46678/0 (socket says 172.21.15.100:46678) 2023-03-24T22:24:40.521 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.518+0000 7ffa16688700 1 -- 172.21.15.100:0/110721120 learned_addr learned my addr 172.21.15.100:0/110721120 (peer_addr_for_me v1:172.21.15.100:0/0) 2023-03-24T22:24:40.521 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.518+0000 7ffa15e87700 1 --2- 172.21.15.100:0/110721120 >> v2:172.21.15.18:3300/0 conn(0x7ffa101610d0 0x7ffa100aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.521 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.519+0000 7ffa14e85700 1 -- 172.21.15.100:0/110721120 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262862038 0 0) 0x7ffa1015d0a0 con 0x7ffa10071680 2023-03-24T22:24:40.521 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.519+0000 7ffa14e85700 1 -- 172.21.15.100:0/110721120 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9fc0035b0 con 0x7ffa10071680 2023-03-24T22:24:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.519+0000 7ffa15e87700 1 -- 172.21.15.100:0/110721120 >> v1:172.21.15.100:6789/0 conn(0x7ffa10071680 legacy=0x7ffa1006a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.519+0000 7ffa15e87700 1 -- 172.21.15.100:0/110721120 >> v1:172.21.15.18:6789/0 conn(0x7ffa101614a0 legacy=0x7ffa10070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.519+0000 7ffa15e87700 1 -- 172.21.15.100:0/110721120 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1006e010 con 0x7ffa101610d0 2023-03-24T22:24:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.519+0000 7ffa15e87700 1 --2- 172.21.15.100:0/110721120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa101610d0 0x7ffa100aeab0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009840 tx=0x7ffa0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e16c1c4f7bc8485 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.519+0000 7ffa14e85700 1 -- 172.21.15.100:0/110721120 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c011830 con 0x7ffa101610d0 2023-03-24T22:24:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.519+0000 7ffa14e85700 1 -- 172.21.15.100:0/110721120 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c011990 con 0x7ffa101610d0 2023-03-24T22:24:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.519+0000 7ffa14e85700 1 -- 172.21.15.100:0/110721120 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c011d20 con 0x7ffa101610d0 2023-03-24T22:24:40.525 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.520+0000 7ffa180eb700 1 -- 172.21.15.100:0/110721120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa101610d0 msgr2=0x7ffa100aeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.525 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.520+0000 7ffa180eb700 1 --2- 172.21.15.100:0/110721120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa101610d0 0x7ffa100aeab0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009840 tx=0x7ffa0c009b50 comp rx=0 tx=0).stop 2023-03-24T22:24:40.525 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.520+0000 7ffa180eb700 1 -- 172.21.15.100:0/110721120 shutdown_connections 2023-03-24T22:24:40.525 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.520+0000 7ffa180eb700 1 --2- 172.21.15.100:0/110721120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa101610d0 0x7ffa100aeab0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.525 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.520+0000 7ffa180eb700 1 -- 172.21.15.100:0/110721120 >> 172.21.15.100:0/110721120 conn(0x7ffa100d0a00 msgr2=0x7ffa100cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:40.525 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.520+0000 7ffa180eb700 1 -- 172.21.15.100:0/110721120 shutdown_connections 2023-03-24T22:24:40.526 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.520+0000 7ffa180eb700 1 -- 172.21.15.100:0/110721120 wait complete. 2023-03-24T22:24:40.526 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.521+0000 7ffa180eb700 1 Processor -- start 2023-03-24T22:24:40.526 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.521+0000 7ffa180eb700 1 -- start start 2023-03-24T22:24:40.526 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa180eb700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa10071680 0x7ffa100b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.526 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa180eb700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa101610d0 0x7ffa1015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.526 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa180eb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa101614a0 0x7ffa10153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.526 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa180eb700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1015e9c0 con 0x7ffa101610d0 2023-03-24T22:24:40.526 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa180eb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1015eca0 con 0x7ffa101614a0 2023-03-24T22:24:40.527 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa180eb700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1015ef80 con 0x7ffa10071680 2023-03-24T22:24:40.527 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa15e87700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa10071680 0x7ffa100b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.527 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa15e87700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa10071680 0x7ffa100b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:34110/0 (socket says 172.21.15.100:34110) 2023-03-24T22:24:40.527 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa16688700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa101614a0 0x7ffa10153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.527 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa15e87700 1 -- 172.21.15.100:0/1612646490 learned_addr learned my addr 172.21.15.100:0/1612646490 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:40.527 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa15686700 1 --2- 172.21.15.100:0/1612646490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa101610d0 0x7ffa1015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.527 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa15e87700 1 -- 172.21.15.100:0/1612646490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa101614a0 msgr2=0x7ffa10153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.528 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa15e87700 1 --2- 172.21.15.100:0/1612646490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa101614a0 0x7ffa10153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.528 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa15e87700 1 -- 172.21.15.100:0/1612646490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa101610d0 msgr2=0x7ffa1015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.528 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa15e87700 1 --2- 172.21.15.100:0/1612646490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa101610d0 0x7ffa1015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.528 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa15e87700 1 -- 172.21.15.100:0/1612646490 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c0094a0 con 0x7ffa10071680 2023-03-24T22:24:40.528 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa15e87700 1 --2- 172.21.15.100:0/1612646490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa10071680 0x7ffa100b33c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0090b0 tx=0x7ffa0c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:40.528 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa067fc700 1 -- 172.21.15.100:0/1612646490 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0036a0 con 0x7ffa10071680 2023-03-24T22:24:40.528 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa067fc700 1 -- 172.21.15.100:0/1612646490 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c01f070 con 0x7ffa10071680 2023-03-24T22:24:40.528 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.522+0000 7ffa180eb700 1 -- 172.21.15.100:0/1612646490 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1006e5d0 con 0x7ffa10071680 2023-03-24T22:24:40.529 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.523+0000 7ffa067fc700 1 -- 172.21.15.100:0/1612646490 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c003a20 con 0x7ffa10071680 2023-03-24T22:24:40.529 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.523+0000 7ffa180eb700 1 -- 172.21.15.100:0/1612646490 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1006e2f0 con 0x7ffa10071680 2023-03-24T22:24:40.529 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.523+0000 7ffa067fc700 1 -- 172.21.15.100:0/1612646490 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffa0c026480 con 0x7ffa10071680 2023-03-24T22:24:40.529 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.523+0000 7ffa067fc700 1 -- 172.21.15.100:0/1612646490 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7ffa0c021070 con 0x7ffa10071680 2023-03-24T22:24:40.529 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.523+0000 7ffa180eb700 1 -- 172.21.15.100:0/1612646490 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1006dcc0 con 0x7ffa10071680 2023-03-24T22:24:40.529 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.526+0000 7ffa067fc700 1 -- 172.21.15.100:0/1612646490 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7ffa0c015020 con 0x7ffa10071680 2023-03-24T22:24:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.608+0000 7ffa180eb700 1 -- 172.21.15.100:0/1612646490 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2a8e5486-0ae6-43c3-870e-07ad88a43340", "id": 3} v 0) v1 -- 0x7ffa100722b0 con 0x7ffa10071680 2023-03-24T22:24:40.612 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.611+0000 7ffa067fc700 1 -- 172.21.15.100:0/1612646490 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a8e5486-0ae6-43c3-870e-07ad88a43340", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffa0c026940 con 0x7ffa10071680 2023-03-24T22:24:40.613 INFO:teuthology.orchestra.run.smithi100.stdout:3 2023-03-24T22:24:40.614 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.612+0000 7ffa180eb700 1 -- 172.21.15.100:0/1612646490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa10071680 msgr2=0x7ffa100b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.614 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.612+0000 7ffa180eb700 1 --2- 172.21.15.100:0/1612646490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa10071680 0x7ffa100b33c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0090b0 tx=0x7ffa0c004750 comp rx=0 tx=0).stop 2023-03-24T22:24:40.614 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.612+0000 7ffa180eb700 1 -- 172.21.15.100:0/1612646490 shutdown_connections 2023-03-24T22:24:40.614 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.612+0000 7ffa180eb700 1 --2- 172.21.15.100:0/1612646490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa101614a0 0x7ffa10153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.614 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.612+0000 7ffa180eb700 1 --2- 172.21.15.100:0/1612646490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa10071680 0x7ffa100b33c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.614 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.612+0000 7ffa180eb700 1 --2- 172.21.15.100:0/1612646490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa101610d0 0x7ffa1015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.614 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.612+0000 7ffa180eb700 1 -- 172.21.15.100:0/1612646490 >> 172.21.15.100:0/1612646490 conn(0x7ffa100d0a00 msgr2=0x7ffa10070f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:40.615 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.612+0000 7ffa180eb700 1 -- 172.21.15.100:0/1612646490 shutdown_connections 2023-03-24T22:24:40.615 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.612+0000 7ffa180eb700 1 -- 172.21.15.100:0/1612646490 wait complete. 2023-03-24T22:24:40.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 9a82df25-c818-47d7-8a9e-3e6fd55cd4f3 4 2023-03-24T22:24:40.749 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.747+0000 7f7f33798700 1 Processor -- start 2023-03-24T22:24:40.749 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.748+0000 7f7f33798700 1 -- start start 2023-03-24T22:24:40.750 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.748+0000 7f7f33798700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15ad10 0x7f7f2c0c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.750 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.748+0000 7f7f33798700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f7f2c0cbb70 0x7f7f2c0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.751 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.748+0000 7f7f33798700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2c066140 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.751 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.748+0000 7f7f33798700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f2c0b55c0 con 0x7f7f2c0cbb70 2023-03-24T22:24:40.751 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.748+0000 7f7f33798700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c0b5080 con 0x7f7f2c15ad10 2023-03-24T22:24:40.751 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f31534700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15ad10 0x7f7f2c0c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.751 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f31534700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15ad10 0x7f7f2c0c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:34114/0 (socket says 172.21.15.100:34114) 2023-03-24T22:24:40.751 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f31534700 1 -- 172.21.15.100:0/3203596746 learned_addr learned my addr 172.21.15.100:0/3203596746 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:40.751 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f30d33700 1 --2- 172.21.15.100:0/3203596746 >> v2:172.21.15.18:3300/0 conn(0x7f7f2c0cbb70 0x7f7f2c0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.752 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f31534700 1 -- 172.21.15.100:0/3203596746 >> v2:172.21.15.18:3300/0 conn(0x7f7f2c0cbb70 msgr2=0x7f7f2c0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:40.752 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f31534700 1 --2- 172.21.15.100:0/3203596746 >> v2:172.21.15.18:3300/0 conn(0x7f7f2c0cbb70 0x7f7f2c0b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.752 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f31534700 1 -- 172.21.15.100:0/3203596746 >> v1:172.21.15.18:6789/0 conn(0x7f7f2c0b4c60 legacy=0x7f7f2c06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.752 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f31534700 1 -- 172.21.15.100:0/3203596746 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c15b0e0 con 0x7f7f2c15ad10 2023-03-24T22:24:40.752 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f31534700 1 --2- 172.21.15.100:0/3203596746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15ad10 0x7f7f2c0c6670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7f280098f0 tx=0x7f7f28009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99397b3bb4afbc0f server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:40.752 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f23fff700 1 -- 172.21.15.100:0/3203596746 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28011830 con 0x7f7f2c15ad10 2023-03-24T22:24:40.752 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f23fff700 1 -- 172.21.15.100:0/3203596746 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f28011990 con 0x7f7f2c15ad10 2023-03-24T22:24:40.753 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.749+0000 7f7f23fff700 1 -- 172.21.15.100:0/3203596746 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28011d20 con 0x7f7f2c15ad10 2023-03-24T22:24:40.755 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.750+0000 7f7f33798700 1 -- 172.21.15.100:0/3203596746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15ad10 msgr2=0x7f7f2c0c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.755 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.750+0000 7f7f33798700 1 --2- 172.21.15.100:0/3203596746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15ad10 0x7f7f2c0c6670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7f280098f0 tx=0x7f7f28009c00 comp rx=0 tx=0).stop 2023-03-24T22:24:40.755 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.750+0000 7f7f33798700 1 -- 172.21.15.100:0/3203596746 shutdown_connections 2023-03-24T22:24:40.755 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.750+0000 7f7f33798700 1 --2- 172.21.15.100:0/3203596746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15ad10 0x7f7f2c0c6670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.756 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.750+0000 7f7f33798700 1 --2- 172.21.15.100:0/3203596746 >> v2:172.21.15.18:3300/0 conn(0x7f7f2c0cbb70 0x7f7f2c0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.756 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.750+0000 7f7f33798700 1 -- 172.21.15.100:0/3203596746 >> 172.21.15.100:0/3203596746 conn(0x7f7f2c06d5a0 msgr2=0x7f7f2c0cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:40.756 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.750+0000 7f7f33798700 1 -- 172.21.15.100:0/3203596746 shutdown_connections 2023-03-24T22:24:40.756 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.750+0000 7f7f33798700 1 -- 172.21.15.100:0/3203596746 wait complete. 2023-03-24T22:24:40.756 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.751+0000 7f7f33798700 1 Processor -- start 2023-03-24T22:24:40.756 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.751+0000 7f7f33798700 1 -- start start 2023-03-24T22:24:40.756 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f33798700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c0b4c60 0x7f7f2c0c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.756 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f33798700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2c0cbb70 0x7f7f2c0c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f33798700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f2c15ad10 0x7f7f2c1e3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f33798700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c1e4180 con 0x7f7f2c0cbb70 2023-03-24T22:24:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f33798700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c1e4460 con 0x7f7f2c15ad10 2023-03-24T22:24:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f33798700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c1e4740 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f31534700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c0b4c60 0x7f7f2c0c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f31534700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c0b4c60 0x7f7f2c0c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:34118/0 (socket says 172.21.15.100:34118) 2023-03-24T22:24:40.757 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f31534700 1 -- 172.21.15.100:0/179315532 learned_addr learned my addr 172.21.15.100:0/179315532 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f31d35700 1 --2- 172.21.15.100:0/179315532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f2c15ad10 0x7f7f2c1e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f30d33700 1 --2- 172.21.15.100:0/179315532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2c0cbb70 0x7f7f2c0c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f31534700 1 -- 172.21.15.100:0/179315532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f2c15ad10 msgr2=0x7f7f2c1e3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f31534700 1 --2- 172.21.15.100:0/179315532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f2c15ad10 0x7f7f2c1e3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f31534700 1 -- 172.21.15.100:0/179315532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2c0cbb70 msgr2=0x7f7f2c0c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f31534700 1 --2- 172.21.15.100:0/179315532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2c0cbb70 0x7f7f2c0c35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f31534700 1 -- 172.21.15.100:0/179315532 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f28009550 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.758 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f31534700 1 --2- 172.21.15.100:0/179315532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c0b4c60 0x7f7f2c0c30b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7f28000c00 tx=0x7f7f28004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f21ffb700 1 -- 172.21.15.100:0/179315532 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28004400 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f21ffb700 1 -- 172.21.15.100:0/179315532 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2801f070 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.752+0000 7f7f33798700 1 -- 172.21.15.100:0/179315532 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2c066670 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.753+0000 7f7f21ffb700 1 -- 172.21.15.100:0/179315532 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f280038e0 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.753+0000 7f7f33798700 1 -- 172.21.15.100:0/179315532 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2c066390 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.753+0000 7f7f21ffb700 1 -- 172.21.15.100:0/179315532 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7f28021030 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.753+0000 7f7f21ffb700 1 -- 172.21.15.100:0/179315532 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f7f2801adc0 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.754+0000 7f7f33798700 1 -- 172.21.15.100:0/179315532 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2c06a7d0 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.760 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.756+0000 7f7f21ffb700 1 -- 172.21.15.100:0/179315532 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f7f28015020 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.839 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.838+0000 7f7f33798700 1 -- 172.21.15.100:0/179315532 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9a82df25-c818-47d7-8a9e-3e6fd55cd4f3", "id": 4} v 0) v1 -- 0x7f7f2c04e850 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.842 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.841+0000 7f7f21ffb700 1 -- 172.21.15.100:0/179315532 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9a82df25-c818-47d7-8a9e-3e6fd55cd4f3", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f2801a450 con 0x7f7f2c0b4c60 2023-03-24T22:24:40.842 INFO:teuthology.orchestra.run.smithi100.stdout:4 2023-03-24T22:24:40.844 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.842+0000 7f7f33798700 1 -- 172.21.15.100:0/179315532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c0b4c60 msgr2=0x7f7f2c0c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.844 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.842+0000 7f7f33798700 1 --2- 172.21.15.100:0/179315532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c0b4c60 0x7f7f2c0c30b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7f28000c00 tx=0x7f7f28004750 comp rx=0 tx=0).stop 2023-03-24T22:24:40.844 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.842+0000 7f7f33798700 1 -- 172.21.15.100:0/179315532 shutdown_connections 2023-03-24T22:24:40.844 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.842+0000 7f7f33798700 1 --2- 172.21.15.100:0/179315532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f2c15ad10 0x7f7f2c1e3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.844 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.842+0000 7f7f33798700 1 --2- 172.21.15.100:0/179315532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c0b4c60 0x7f7f2c0c30b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.844 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.842+0000 7f7f33798700 1 --2- 172.21.15.100:0/179315532 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f2c0cbb70 0x7f7f2c0c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.845 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.842+0000 7f7f33798700 1 -- 172.21.15.100:0/179315532 >> 172.21.15.100:0/179315532 conn(0x7f7f2c06d5a0 msgr2=0x7f7f2c0cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:40.845 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.842+0000 7f7f33798700 1 -- 172.21.15.100:0/179315532 shutdown_connections 2023-03-24T22:24:40.845 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.842+0000 7f7f33798700 1 -- 172.21.15.100:0/179315532 wait complete. 2023-03-24T22:24:40.855 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new ae37e2cd-d079-4962-9bd7-628aa5a7e14d 5 2023-03-24T22:24:40.983 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.982+0000 7f901aef8700 1 Processor -- start 2023-03-24T22:24:40.983 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.982+0000 7f901aef8700 1 -- start start 2023-03-24T22:24:40.985 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f901aef8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f90141614a0 0x7f9014070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.985 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f901aef8700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f901406c2d0 0x7f90140714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.986 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f901aef8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90141607c0 con 0x7f90141610d0 2023-03-24T22:24:40.986 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f901aef8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901415d0a0 con 0x7f90141614a0 2023-03-24T22:24:40.986 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f901aef8700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90140af500 con 0x7f901406c2d0 2023-03-24T22:24:40.986 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f9019495700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f901406c2d0 0x7f90140714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.986 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f9019495700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f901406c2d0 0x7f90140714b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40914/0 (socket says 172.21.15.100:40914) 2023-03-24T22:24:40.986 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f9019495700 1 -- 172.21.15.100:0/4023082470 learned_addr learned my addr 172.21.15.100:0/4023082470 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f9013fff700 1 --2- 172.21.15.100:0/4023082470 >> v2:172.21.15.18:3300/0 conn(0x7f90141614a0 0x7f9014070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f9019495700 1 -- 172.21.15.100:0/4023082470 >> v2:172.21.15.18:3300/0 conn(0x7f90141614a0 msgr2=0x7f9014070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f9019495700 1 --2- 172.21.15.100:0/4023082470 >> v2:172.21.15.18:3300/0 conn(0x7f90141614a0 0x7f9014070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f9019495700 1 -- 172.21.15.100:0/4023082470 >> v1:172.21.15.18:6789/0 conn(0x7f90141610d0 legacy=0x7f90140aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f9019495700 1 -- 172.21.15.100:0/4023082470 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901406e010 con 0x7f901406c2d0 2023-03-24T22:24:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.983+0000 7f9019495700 1 --2- 172.21.15.100:0/4023082470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901406c2d0 0x7f90140714b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f90140af7e0 tx=0x7f900c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40808266e7e16de3 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:40.987 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f90137fe700 1 -- 172.21.15.100:0/4023082470 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c00e070 con 0x7f901406c2d0 2023-03-24T22:24:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f90137fe700 1 -- 172.21.15.100:0/4023082470 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c00b910 con 0x7f901406c2d0 2023-03-24T22:24:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f90137fe700 1 -- 172.21.15.100:0/4023082470 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c00bcc0 con 0x7f901406c2d0 2023-03-24T22:24:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f901aef8700 1 -- 172.21.15.100:0/4023082470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901406c2d0 msgr2=0x7f90140714b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f901aef8700 1 --2- 172.21.15.100:0/4023082470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901406c2d0 0x7f90140714b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f90140af7e0 tx=0x7f900c00b330 comp rx=0 tx=0).stop 2023-03-24T22:24:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f901aef8700 1 -- 172.21.15.100:0/4023082470 shutdown_connections 2023-03-24T22:24:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f901aef8700 1 --2- 172.21.15.100:0/4023082470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901406c2d0 0x7f90140714b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.988 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f901aef8700 1 --2- 172.21.15.100:0/4023082470 >> v2:172.21.15.18:3300/0 conn(0x7f90141614a0 0x7f9014070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.989 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f901aef8700 1 -- 172.21.15.100:0/4023082470 >> 172.21.15.100:0/4023082470 conn(0x7f90140d0a00 msgr2=0x7f90140cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:40.989 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f901aef8700 1 -- 172.21.15.100:0/4023082470 shutdown_connections 2023-03-24T22:24:40.989 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.984+0000 7f901aef8700 1 -- 172.21.15.100:0/4023082470 wait complete. 2023-03-24T22:24:40.991 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.985+0000 7f901aef8700 1 Processor -- start 2023-03-24T22:24:40.991 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.986+0000 7f901aef8700 1 -- start start 2023-03-24T22:24:40.991 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.986+0000 7f901aef8700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f901406c2d0 0x7f9014153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.991 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.986+0000 7f901aef8700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90141610d0 0x7f901415f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.992 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.986+0000 7f901aef8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90141614a0 0x7f901415dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:40.992 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.986+0000 7f901aef8700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901415fb20 con 0x7f901406c2d0 2023-03-24T22:24:40.992 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.986+0000 7f901aef8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901415f150 con 0x7f90141614a0 2023-03-24T22:24:40.992 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.986+0000 7f901aef8700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901415e540 con 0x7f90141610d0 2023-03-24T22:24:40.992 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9013fff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90141610d0 0x7f901415f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.992 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9019495700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90141614a0 0x7f901415dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.992 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9013fff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90141610d0 0x7f901415f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:34124/0 (socket says 172.21.15.100:34124) 2023-03-24T22:24:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9013fff700 1 -- 172.21.15.100:0/4019777894 learned_addr learned my addr 172.21.15.100:0/4019777894 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9018c94700 1 --2- 172.21.15.100:0/4019777894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f901406c2d0 0x7f9014153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9013fff700 1 -- 172.21.15.100:0/4019777894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90141614a0 msgr2=0x7f901415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9013fff700 1 --2- 172.21.15.100:0/4019777894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90141614a0 0x7f901415dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9013fff700 1 -- 172.21.15.100:0/4019777894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f901406c2d0 msgr2=0x7f9014153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:40.993 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9013fff700 1 --2- 172.21.15.100:0/4019777894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f901406c2d0 0x7f9014153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9013fff700 1 -- 172.21.15.100:0/4019777894 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c009b30 con 0x7f90141610d0 2023-03-24T22:24:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9018c94700 1 --2- 172.21.15.100:0/4019777894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f901406c2d0 0x7f9014153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9019495700 1 --2- 172.21.15.100:0/4019777894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90141614a0 0x7f901415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f9013fff700 1 --2- 172.21.15.100:0/4019777894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90141610d0 0x7f901415f5e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f900000e870 tx=0x7f900000eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f90117fa700 1 -- 172.21.15.100:0/4019777894 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000013680 con 0x7f90141610d0 2023-03-24T22:24:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f90117fa700 1 -- 172.21.15.100:0/4019777894 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90000137e0 con 0x7f90141610d0 2023-03-24T22:24:40.994 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f901aef8700 1 -- 172.21.15.100:0/4019777894 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901406e5d0 con 0x7f90141610d0 2023-03-24T22:24:40.995 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f90117fa700 1 -- 172.21.15.100:0/4019777894 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000013b70 con 0x7f90141610d0 2023-03-24T22:24:40.995 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.987+0000 7f901aef8700 1 -- 172.21.15.100:0/4019777894 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901406e2f0 con 0x7f90141610d0 2023-03-24T22:24:40.995 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.988+0000 7f90117fa700 1 -- 172.21.15.100:0/4019777894 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9000005a20 con 0x7f90141610d0 2023-03-24T22:24:40.995 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.988+0000 7f901aef8700 1 -- 172.21.15.100:0/4019777894 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90140af1c0 con 0x7f90141610d0 2023-03-24T22:24:40.995 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.988+0000 7f90117fa700 1 -- 172.21.15.100:0/4019777894 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9000006700 con 0x7f90141610d0 2023-03-24T22:24:40.995 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:40.990+0000 7f90117fa700 1 -- 172.21.15.100:0/4019777894 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f9000021060 con 0x7f90141610d0 2023-03-24T22:24:41.075 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.074+0000 7f901aef8700 1 -- 172.21.15.100:0/4019777894 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ae37e2cd-d079-4962-9bd7-628aa5a7e14d", "id": 5} v 0) v1 -- 0x7f901404e850 con 0x7f90141610d0 2023-03-24T22:24:41.079 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.078+0000 7f90117fa700 1 -- 172.21.15.100:0/4019777894 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae37e2cd-d079-4962-9bd7-628aa5a7e14d", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9000017070 con 0x7f90141610d0 2023-03-24T22:24:41.080 INFO:teuthology.orchestra.run.smithi100.stdout:5 2023-03-24T22:24:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.079+0000 7f901aef8700 1 -- 172.21.15.100:0/4019777894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90141610d0 msgr2=0x7f901415f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.079+0000 7f901aef8700 1 --2- 172.21.15.100:0/4019777894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90141610d0 0x7f901415f5e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f900000e870 tx=0x7f900000eb80 comp rx=0 tx=0).stop 2023-03-24T22:24:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.079+0000 7f901aef8700 1 -- 172.21.15.100:0/4019777894 shutdown_connections 2023-03-24T22:24:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.079+0000 7f901aef8700 1 --2- 172.21.15.100:0/4019777894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f901406c2d0 0x7f9014153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.081 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.079+0000 7f901aef8700 1 --2- 172.21.15.100:0/4019777894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90141614a0 0x7f901415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.079+0000 7f901aef8700 1 --2- 172.21.15.100:0/4019777894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90141610d0 0x7f901415f5e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.079+0000 7f901aef8700 1 -- 172.21.15.100:0/4019777894 >> 172.21.15.100:0/4019777894 conn(0x7f90140d0a00 msgr2=0x7f90140ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.079+0000 7f901aef8700 1 -- 172.21.15.100:0/4019777894 shutdown_connections 2023-03-24T22:24:41.082 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.079+0000 7f901aef8700 1 -- 172.21.15.100:0/4019777894 wait complete. 2023-03-24T22:24:41.097 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new e3574199-2966-431b-ab0b-25efed52c749 6 2023-03-24T22:24:41.236 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.235+0000 7f42f12f8700 1 Processor -- start 2023-03-24T22:24:41.236 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.235+0000 7f42f12f8700 1 -- start start 2023-03-24T22:24:41.238 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42f12f8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f42ec15cd30 0x7f42ec0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.238 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42f12f8700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f42ec0c9f20 0x7f42ec0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.238 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42f12f8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec071220 con 0x7f42ec071890 2023-03-24T22:24:41.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42f12f8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec06be90 con 0x7f42ec15cd30 2023-03-24T22:24:41.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42f12f8700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec0b2c20 con 0x7f42ec0c9f20 2023-03-24T22:24:41.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42eb7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f42ec0c9f20 0x7f42ec0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42eb7fe700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f42ec0c9f20 0x7f42ec0ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46904/0 (socket says 172.21.15.100:46904) 2023-03-24T22:24:41.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42eb7fe700 1 -- 172.21.15.100:0/1022490722 learned_addr learned my addr 172.21.15.100:0/1022490722 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:41.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42eaffd700 1 --2- 172.21.15.100:0/1022490722 >> v2:172.21.15.18:3300/0 conn(0x7f42ec15cd30 0x7f42ec0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42eb7fe700 1 -- 172.21.15.100:0/1022490722 >> v2:172.21.15.18:3300/0 conn(0x7f42ec15cd30 msgr2=0x7f42ec0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42eb7fe700 1 --2- 172.21.15.100:0/1022490722 >> v2:172.21.15.18:3300/0 conn(0x7f42ec15cd30 0x7f42ec0d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42eb7fe700 1 -- 172.21.15.100:0/1022490722 >> v1:172.21.15.18:6789/0 conn(0x7f42ec071890 legacy=0x7f42ec0b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42eb7fe700 1 -- 172.21.15.100:0/1022490722 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0ca8a0 con 0x7f42ec0c9f20 2023-03-24T22:24:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42eb7fe700 1 --2- 172.21.15.100:0/1022490722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42ec0c9f20 0x7f42ec0ca360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42ec070660 tx=0x7f42e4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b60831a7cfca1a79 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.236+0000 7f42e9ffb700 1 -- 172.21.15.100:0/1022490722 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4011830 con 0x7f42ec0c9f20 2023-03-24T22:24:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.237+0000 7f42e9ffb700 1 -- 172.21.15.100:0/1022490722 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e4011990 con 0x7f42ec0c9f20 2023-03-24T22:24:41.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.237+0000 7f42e9ffb700 1 -- 172.21.15.100:0/1022490722 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4011d20 con 0x7f42ec0c9f20 2023-03-24T22:24:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.237+0000 7f42f12f8700 1 -- 172.21.15.100:0/1022490722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42ec0c9f20 msgr2=0x7f42ec0ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.237+0000 7f42f12f8700 1 --2- 172.21.15.100:0/1022490722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42ec0c9f20 0x7f42ec0ca360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42ec070660 tx=0x7f42e4009bd0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.237+0000 7f42f12f8700 1 -- 172.21.15.100:0/1022490722 shutdown_connections 2023-03-24T22:24:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.237+0000 7f42f12f8700 1 --2- 172.21.15.100:0/1022490722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42ec0c9f20 0x7f42ec0ca360 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.237+0000 7f42f12f8700 1 --2- 172.21.15.100:0/1022490722 >> v2:172.21.15.18:3300/0 conn(0x7f42ec15cd30 0x7f42ec0d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.237+0000 7f42f12f8700 1 -- 172.21.15.100:0/1022490722 >> 172.21.15.100:0/1022490722 conn(0x7f42ec06d5a0 msgr2=0x7f42ec0cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.237+0000 7f42f12f8700 1 -- 172.21.15.100:0/1022490722 shutdown_connections 2023-03-24T22:24:41.241 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.237+0000 7f42f12f8700 1 -- 172.21.15.100:0/1022490722 wait complete. 2023-03-24T22:24:41.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.238+0000 7f42f12f8700 1 Processor -- start 2023-03-24T22:24:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42f12f8700 1 -- start start 2023-03-24T22:24:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42f12f8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42ec071890 0x7f42ec15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42f12f8700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42ec0c9f20 0x7f42ec156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42f12f8700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42ec15cd30 0x7f42ec15d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42f12f8700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0c7a90 con 0x7f42ec0c9f20 2023-03-24T22:24:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42f12f8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec15d950 con 0x7f42ec071890 2023-03-24T22:24:41.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42f12f8700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec15dc30 con 0x7f42ec15cd30 2023-03-24T22:24:41.245 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42eb7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42ec15cd30 0x7f42ec15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.245 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42eb7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42ec15cd30 0x7f42ec15d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:33284/0 (socket says 172.21.15.100:33284) 2023-03-24T22:24:41.245 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42eb7fe700 1 -- 172.21.15.100:0/3517136383 learned_addr learned my addr 172.21.15.100:0/3517136383 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:41.245 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42eaffd700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42ec071890 0x7f42ec15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.245 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.239+0000 7f42ea7fc700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42ec0c9f20 0x7f42ec156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.245 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42eb7fe700 1 -- 172.21.15.100:0/3517136383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42ec071890 msgr2=0x7f42ec15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.245 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42eb7fe700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42ec071890 0x7f42ec15fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.246 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42eb7fe700 1 -- 172.21.15.100:0/3517136383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42ec0c9f20 msgr2=0x7f42ec156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.246 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42eb7fe700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42ec0c9f20 0x7f42ec156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.246 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42eb7fe700 1 -- 172.21.15.100:0/3517136383 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e4009550 con 0x7f42ec15cd30 2023-03-24T22:24:41.246 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42eaffd700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42ec071890 0x7f42ec15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:41.247 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42ea7fc700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42ec0c9f20 0x7f42ec156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:41.247 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42eb7fe700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42ec15cd30 0x7f42ec15d100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009c10 tx=0x7f42e4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:41.247 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42e37fe700 1 -- 172.21.15.100:0/3517136383 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4004240 con 0x7f42ec15cd30 2023-03-24T22:24:41.247 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42e37fe700 1 -- 172.21.15.100:0/3517136383 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e401f070 con 0x7f42ec15cd30 2023-03-24T22:24:41.247 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42e37fe700 1 -- 172.21.15.100:0/3517136383 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e40045e0 con 0x7f42ec15cd30 2023-03-24T22:24:41.248 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42f12f8700 1 -- 172.21.15.100:0/3517136383 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ec06b840 con 0x7f42ec15cd30 2023-03-24T22:24:41.248 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42f12f8700 1 -- 172.21.15.100:0/3517136383 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec071470 con 0x7f42ec15cd30 2023-03-24T22:24:41.248 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42e37fe700 1 -- 172.21.15.100:0/3517136383 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f42e4021030 con 0x7f42ec15cd30 2023-03-24T22:24:41.248 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.240+0000 7f42e37fe700 1 -- 172.21.15.100:0/3517136383 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f42e401acc0 con 0x7f42ec15cd30 2023-03-24T22:24:41.248 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.241+0000 7f42f12f8700 1 -- 172.21.15.100:0/3517136383 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ec06bb20 con 0x7f42ec15cd30 2023-03-24T22:24:41.249 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.243+0000 7f42e37fe700 1 -- 172.21.15.100:0/3517136383 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f42e4015020 con 0x7f42ec15cd30 2023-03-24T22:24:41.256 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:41.255+0000 7f48f8982200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-24T22:24:41.274 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:41.272+0000 7f1e5e1b4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-24T22:24:41.327 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.326+0000 7f42f12f8700 1 -- 172.21.15.100:0/3517136383 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e3574199-2966-431b-ab0b-25efed52c749", "id": 6} v 0) v1 -- 0x7f42ec04e850 con 0x7f42ec15cd30 2023-03-24T22:24:41.330 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.329+0000 7f42e37fe700 1 -- 172.21.15.100:0/3517136383 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e3574199-2966-431b-ab0b-25efed52c749", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f42e4003c00 con 0x7f42ec15cd30 2023-03-24T22:24:41.330 INFO:teuthology.orchestra.run.smithi100.stdout:6 2023-03-24T22:24:41.332 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.330+0000 7f42f12f8700 1 -- 172.21.15.100:0/3517136383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42ec15cd30 msgr2=0x7f42ec15d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.332 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.330+0000 7f42f12f8700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42ec15cd30 0x7f42ec15d100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009c10 tx=0x7f42e4004750 comp rx=0 tx=0).stop 2023-03-24T22:24:41.332 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.330+0000 7f42f12f8700 1 -- 172.21.15.100:0/3517136383 shutdown_connections 2023-03-24T22:24:41.333 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.330+0000 7f42f12f8700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42ec15cd30 0x7f42ec15d100 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.333 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.330+0000 7f42f12f8700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42ec071890 0x7f42ec15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.333 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.330+0000 7f42f12f8700 1 --2- 172.21.15.100:0/3517136383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42ec0c9f20 0x7f42ec156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.333 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.330+0000 7f42f12f8700 1 -- 172.21.15.100:0/3517136383 >> 172.21.15.100:0/3517136383 conn(0x7f42ec06d5a0 msgr2=0x7f42ec06a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:41.336 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.331+0000 7f42f12f8700 1 -- 172.21.15.100:0/3517136383 shutdown_connections 2023-03-24T22:24:41.336 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.331+0000 7f42f12f8700 1 -- 172.21.15.100:0/3517136383 wait complete. 2023-03-24T22:24:41.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 5a2a90ec-837f-4560-8a8e-b3e82eab590b 7 2023-03-24T22:24:41.406 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:41.405+0000 7f48f8982200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-24T22:24:41.422 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:41.421+0000 7f1e5e1b4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-24T22:24:41.465 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.464+0000 7f8265bbc700 1 Processor -- start 2023-03-24T22:24:41.465 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.464+0000 7f8265bbc700 1 -- start start 2023-03-24T22:24:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.464+0000 7f8265bbc700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f82601610d0 0x7f82600aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.464+0000 7f8265bbc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82601614a0 0x7f8260070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.464+0000 7f8265bbc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f826006c2d0 0x7f82600714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.464+0000 7f8265bbc700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82601607c0 con 0x7f82601610d0 2023-03-24T22:24:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.464+0000 7f8265bbc700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826015d0a0 con 0x7f826006c2d0 2023-03-24T22:24:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.464+0000 7f8265bbc700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82600af500 con 0x7f82601614a0 2023-03-24T22:24:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825ffff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f826006c2d0 0x7f82600714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825effd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82601614a0 0x7f8260070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825effd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82601614a0 0x7f8260070f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:33300/0 (socket says 172.21.15.100:33300) 2023-03-24T22:24:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825ffff700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f826006c2d0 0x7f82600714b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46926/0 (socket says 172.21.15.100:46926) 2023-03-24T22:24:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825effd700 1 -- 172.21.15.100:0/957513280 learned_addr learned my addr 172.21.15.100:0/957513280 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825f7fe700 1 --2- 172.21.15.100:0/957513280 >> v2:172.21.15.18:3300/0 conn(0x7f82601610d0 0x7f82600aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825effd700 1 -- 172.21.15.100:0/957513280 >> v2:172.21.15.100:3300/0 conn(0x7f826006c2d0 msgr2=0x7f82600714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825effd700 1 --2- 172.21.15.100:0/957513280 >> v2:172.21.15.100:3300/0 conn(0x7f826006c2d0 0x7f82600714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825effd700 1 -- 172.21.15.100:0/957513280 >> v2:172.21.15.18:3300/0 conn(0x7f82601610d0 msgr2=0x7f82600aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825effd700 1 --2- 172.21.15.100:0/957513280 >> v2:172.21.15.18:3300/0 conn(0x7f82601610d0 0x7f82600aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825effd700 1 -- 172.21.15.100:0/957513280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826006e010 con 0x7f82601614a0 2023-03-24T22:24:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825ffff700 1 --2- 172.21.15.100:0/957513280 >> v2:172.21.15.100:3300/0 conn(0x7f826006c2d0 0x7f82600714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:41.470 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825effd700 1 --2- 172.21.15.100:0/957513280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82601614a0 0x7f8260070f70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8260072be0 tx=0x7f8254009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4382ea1f81d790 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:41.470 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825e7fc700 1 -- 172.21.15.100:0/957513280 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254011830 con 0x7f82601614a0 2023-03-24T22:24:41.470 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825e7fc700 1 -- 172.21.15.100:0/957513280 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8254011990 con 0x7f82601614a0 2023-03-24T22:24:41.470 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.465+0000 7f825e7fc700 1 -- 172.21.15.100:0/957513280 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254011d20 con 0x7f82601614a0 2023-03-24T22:24:41.470 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.466+0000 7f8265bbc700 1 -- 172.21.15.100:0/957513280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82601614a0 msgr2=0x7f8260070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.470 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.466+0000 7f8265bbc700 1 --2- 172.21.15.100:0/957513280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82601614a0 0x7f8260070f70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8260072be0 tx=0x7f8254009bd0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.470 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.466+0000 7f8265bbc700 1 -- 172.21.15.100:0/957513280 shutdown_connections 2023-03-24T22:24:41.470 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.466+0000 7f8265bbc700 1 --2- 172.21.15.100:0/957513280 >> v2:172.21.15.100:3300/0 conn(0x7f826006c2d0 0x7f82600714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.471 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.466+0000 7f8265bbc700 1 --2- 172.21.15.100:0/957513280 >> v2:172.21.15.18:3300/0 conn(0x7f82601610d0 0x7f82600aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.471 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.466+0000 7f8265bbc700 1 --2- 172.21.15.100:0/957513280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82601614a0 0x7f8260070f70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.471 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.466+0000 7f8265bbc700 1 -- 172.21.15.100:0/957513280 >> 172.21.15.100:0/957513280 conn(0x7f82600d0a00 msgr2=0x7f82600cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:41.471 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.466+0000 7f8265bbc700 1 -- 172.21.15.100:0/957513280 shutdown_connections 2023-03-24T22:24:41.471 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.466+0000 7f8265bbc700 1 -- 172.21.15.100:0/957513280 wait complete. 2023-03-24T22:24:41.473 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.467+0000 7f8265bbc700 1 Processor -- start 2023-03-24T22:24:41.473 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.467+0000 7f8265bbc700 1 -- start start 2023-03-24T22:24:41.473 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f8265bbc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f826006c2d0 0x7f82600c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.473 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f8265bbc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82601610d0 0x7f82600b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.474 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f8265bbc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82601614a0 0x7f8260150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.474 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f8265bbc700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600c4710 con 0x7f82601614a0 2023-03-24T22:24:41.474 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f8265bbc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82601f6c20 con 0x7f82601610d0 2023-03-24T22:24:41.474 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f8265bbc700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82601f60a0 con 0x7f826006c2d0 2023-03-24T22:24:41.474 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f825f7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f826006c2d0 0x7f82600c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.474 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f825f7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f826006c2d0 0x7f82600c41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:33314/0 (socket says 172.21.15.100:33314) 2023-03-24T22:24:41.474 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f825f7fe700 1 -- 172.21.15.100:0/3998504075 learned_addr learned my addr 172.21.15.100:0/3998504075 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-03-24T22:24:41.475 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f825effd700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82601610d0 0x7f82600b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.475 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.468+0000 7f825ffff700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82601614a0 0x7f8260150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.475 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f825f7fe700 1 -- 172.21.15.100:0/3998504075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82601610d0 msgr2=0x7f82600b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.475 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f825f7fe700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82601610d0 0x7f82600b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.475 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f825f7fe700 1 -- 172.21.15.100:0/3998504075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82601614a0 msgr2=0x7f8260150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.475 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f825f7fe700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82601614a0 0x7f8260150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.475 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f825f7fe700 1 -- 172.21.15.100:0/3998504075 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258009a70 con 0x7f826006c2d0 2023-03-24T22:24:41.475 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f825effd700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82601610d0 0x7f82600b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:41.476 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f825ffff700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82601614a0 0x7f8260150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:41.476 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f825f7fe700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f826006c2d0 0x7f82600c41d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f825000e9b0 tx=0x7f825000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:41.476 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f8247fff700 1 -- 172.21.15.100:0/3998504075 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250009710 con 0x7f826006c2d0 2023-03-24T22:24:41.476 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f8247fff700 1 -- 172.21.15.100:0/3998504075 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825000ee40 con 0x7f826006c2d0 2023-03-24T22:24:41.476 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f8265bbc700 1 -- 172.21.15.100:0/3998504075 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8254009550 con 0x7f826006c2d0 2023-03-24T22:24:41.476 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f8247fff700 1 -- 172.21.15.100:0/3998504075 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000ca50 con 0x7f826006c2d0 2023-03-24T22:24:41.476 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f8265bbc700 1 -- 172.21.15.100:0/3998504075 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826006e5d0 con 0x7f826006c2d0 2023-03-24T22:24:41.477 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f8247fff700 1 -- 172.21.15.100:0/3998504075 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8250005a20 con 0x7f826006c2d0 2023-03-24T22:24:41.477 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.469+0000 7f8247fff700 1 -- 172.21.15.100:0/3998504075 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f82500067d0 con 0x7f826006c2d0 2023-03-24T22:24:41.477 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.470+0000 7f8265bbc700 1 -- 172.21.15.100:0/3998504075 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826006e2f0 con 0x7f826006c2d0 2023-03-24T22:24:41.477 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.472+0000 7f8247fff700 1 -- 172.21.15.100:0/3998504075 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f8250021060 con 0x7f826006c2d0 2023-03-24T22:24:41.558 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.556+0000 7f8265bbc700 1 -- 172.21.15.100:0/3998504075 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5a2a90ec-837f-4560-8a8e-b3e82eab590b", "id": 7} v 0) v1 -- 0x7f826006dcc0 con 0x7f826006c2d0 2023-03-24T22:24:41.561 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.560+0000 7f8247fff700 1 -- 172.21.15.100:0/3998504075 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a2a90ec-837f-4560-8a8e-b3e82eab590b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8250017070 con 0x7f826006c2d0 2023-03-24T22:24:41.561 INFO:teuthology.orchestra.run.smithi100.stdout:7 2023-03-24T22:24:41.562 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.560+0000 7f8265bbc700 1 -- 172.21.15.100:0/3998504075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f826006c2d0 msgr2=0x7f82600c41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.562 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.560+0000 7f8265bbc700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f826006c2d0 0x7f82600c41d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f825000e9b0 tx=0x7f825000ecc0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.562 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.561+0000 7f8265bbc700 1 -- 172.21.15.100:0/3998504075 shutdown_connections 2023-03-24T22:24:41.562 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.561+0000 7f8265bbc700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82601614a0 0x7f8260150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.563 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.561+0000 7f8265bbc700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f826006c2d0 0x7f82600c41d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.563 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.561+0000 7f8265bbc700 1 --2- 172.21.15.100:0/3998504075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82601610d0 0x7f82600b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.563 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.561+0000 7f8265bbc700 1 -- 172.21.15.100:0/3998504075 >> 172.21.15.100:0/3998504075 conn(0x7f82600d0a00 msgr2=0x7f82600ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:41.563 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.561+0000 7f8265bbc700 1 -- 172.21.15.100:0/3998504075 shutdown_connections 2023-03-24T22:24:41.563 INFO:teuthology.orchestra.run.smithi100.stderr:2023-03-24T22:24:41.561+0000 7f8265bbc700 1 -- 172.21.15.100:0/3998504075 wait complete. 2023-03-24T22:24:41.574 INFO:tasks.ceph.osd.0:Restarting daemon 2023-03-24T22:24:41.574 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-03-24T22:24:41.577 INFO:tasks.ceph.osd.0:Started 2023-03-24T22:24:41.577 INFO:tasks.ceph.osd.1:Restarting daemon 2023-03-24T22:24:41.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-03-24T22:24:41.580 INFO:tasks.ceph.osd.1:Started 2023-03-24T22:24:41.580 INFO:tasks.ceph.osd.2:Restarting daemon 2023-03-24T22:24:41.581 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-03-24T22:24:41.583 INFO:tasks.ceph.osd.2:Started 2023-03-24T22:24:41.583 INFO:tasks.ceph.osd.3:Restarting daemon 2023-03-24T22:24:41.584 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-03-24T22:24:41.586 INFO:tasks.ceph.osd.3:Started 2023-03-24T22:24:41.587 INFO:tasks.ceph.osd.4:Restarting daemon 2023-03-24T22:24:41.587 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-03-24T22:24:41.589 INFO:tasks.ceph.osd.4:Started 2023-03-24T22:24:41.589 INFO:tasks.ceph.osd.5:Restarting daemon 2023-03-24T22:24:41.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-03-24T22:24:41.591 INFO:tasks.ceph.osd.5:Started 2023-03-24T22:24:41.592 INFO:tasks.ceph.osd.6:Restarting daemon 2023-03-24T22:24:41.592 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-03-24T22:24:41.594 INFO:tasks.ceph.osd.6:Started 2023-03-24T22:24:41.594 INFO:tasks.ceph.osd.7:Restarting daemon 2023-03-24T22:24:41.595 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-03-24T22:24:41.597 INFO:tasks.ceph.osd.7:Started 2023-03-24T22:24:41.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-24T22:24:41.675 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:41.674+0000 7f48f8982200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-24T22:24:41.687 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:41.686+0000 7f1e5e1b4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-24T22:24:41.757 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:41.756+0000 7f48f8982200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-24T22:24:41.763 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:41.762+0000 7f1e5e1b4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-24T22:24:41.764 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.763+0000 7fe7c9bf1700 1 Processor -- start 2023-03-24T22:24:41.764 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.763+0000 7fe7c9bf1700 1 -- start start 2023-03-24T22:24:41.765 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.764+0000 7fe7c9bf1700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe7c406b5b0 0x7fe7c40cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.765 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.764+0000 7fe7c9bf1700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c415e280 con 0x7fe7c4159990 2023-03-24T22:24:41.766 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.764+0000 7fe7c9bf1700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c406bb90 con 0x7fe7c406d200 2023-03-24T22:24:41.766 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.764+0000 7fe7c9bf1700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c40cf1a0 con 0x7fe7c406b5b0 2023-03-24T22:24:41.766 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.764+0000 7fe7c37fe700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fe7c4159990 0x7fe7c415db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:41072/0 (socket says 172.21.15.18:41072) 2023-03-24T22:24:41.766 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.764+0000 7fe7c37fe700 1 -- 172.21.15.18:0/2569794203 learned_addr learned my addr 172.21.15.18:0/2569794203 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:41.766 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c2ffd700 1 --2- 172.21.15.18:0/2569794203 >> v2:172.21.15.100:3300/0 conn(0x7fe7c406b5b0 0x7fe7c40cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.766 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c27fc700 1 -- 172.21.15.18:0/2569794203 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698958912 0 0) 0x7fe7c415e280 con 0x7fe7c4159990 2023-03-24T22:24:41.766 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c27fc700 1 -- 172.21.15.18:0/2569794203 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7ac0057c0 con 0x7fe7c4159990 2023-03-24T22:24:41.770 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c2ffd700 1 -- 172.21.15.18:0/2569794203 >> v1:172.21.15.100:6789/0 conn(0x7fe7c406d200 legacy=0x7fe7c406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.770 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c27fc700 1 -- 172.21.15.18:0/2569794203 <== mon.0 v1:172.21.15.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2525003333 0 0) 0x7fe7ac0057c0 con 0x7fe7c4159990 2023-03-24T22:24:41.770 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c2ffd700 1 -- 172.21.15.18:0/2569794203 >> v1:172.21.15.18:6789/0 conn(0x7fe7c4159990 legacy=0x7fe7c415db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.771 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c2ffd700 1 -- 172.21.15.18:0/2569794203 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c406be70 con 0x7fe7c406b5b0 2023-03-24T22:24:41.771 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c2ffd700 1 --2- 172.21.15.18:0/2569794203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7c406b5b0 0x7fe7c40cec60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009840 tx=0x7fe7b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e065f1610817776 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:41.771 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c27fc700 1 -- 172.21.15.18:0/2569794203 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4001670 con 0x7fe7c406b5b0 2023-03-24T22:24:41.771 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c27fc700 1 -- 172.21.15.18:0/2569794203 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8004030 con 0x7fe7c406b5b0 2023-03-24T22:24:41.771 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.765+0000 7fe7c27fc700 1 -- 172.21.15.18:0/2569794203 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80043c0 con 0x7fe7c406b5b0 2023-03-24T22:24:41.771 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.766+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/2569794203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7c406b5b0 msgr2=0x7fe7c40cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.771 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.766+0000 7fe7c9bf1700 1 --2- 172.21.15.18:0/2569794203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7c406b5b0 0x7fe7c40cec60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009840 tx=0x7fe7b8009b50 comp rx=0 tx=0).stop 2023-03-24T22:24:41.772 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.766+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/2569794203 shutdown_connections 2023-03-24T22:24:41.772 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.766+0000 7fe7c9bf1700 1 --2- 172.21.15.18:0/2569794203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7c406b5b0 0x7fe7c40cec60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.772 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.766+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/2569794203 >> 172.21.15.18:0/2569794203 conn(0x7fe7c40cd590 msgr2=0x7fe7c40cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:41.772 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.766+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/2569794203 shutdown_connections 2023-03-24T22:24:41.772 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.766+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/2569794203 wait complete. 2023-03-24T22:24:41.772 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.767+0000 7fe7c9bf1700 1 Processor -- start 2023-03-24T22:24:41.772 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.767+0000 7fe7c9bf1700 1 -- start start 2023-03-24T22:24:41.773 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c9bf1700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7c406b5b0 0x7fe7c41ee120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.773 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c9bf1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7c406d200 0x7fe7c40c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.773 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c9bf1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c4159990 0x7fe7c40c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:41.773 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c9bf1700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41ee660 con 0x7fe7c4159990 2023-03-24T22:24:41.773 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c9bf1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c40c9d00 con 0x7fe7c406d200 2023-03-24T22:24:41.773 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c9bf1700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c4159e90 con 0x7fe7c406b5b0 2023-03-24T22:24:41.774 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c3fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c4159990 0x7fe7c40c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.774 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7c406d200 0x7fe7c40c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.774 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c37fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7c406b5b0 0x7fe7c41ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:41.774 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c3fff700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c4159990 0x7fe7c40c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56726/0 (socket says 172.21.15.18:56726) 2023-03-24T22:24:41.774 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.768+0000 7fe7c3fff700 1 -- 172.21.15.18:0/555357804 learned_addr learned my addr 172.21.15.18:0/555357804 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:41.774 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7c3fff700 1 -- 172.21.15.18:0/555357804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7c406b5b0 msgr2=0x7fe7c41ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.774 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7c3fff700 1 --2- 172.21.15.18:0/555357804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7c406b5b0 0x7fe7c41ee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.775 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7c3fff700 1 -- 172.21.15.18:0/555357804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7c406d200 msgr2=0x7fe7c40c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.775 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7c3fff700 1 --2- 172.21.15.18:0/555357804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7c406d200 0x7fe7c40c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.775 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7c3fff700 1 -- 172.21.15.18:0/555357804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b80094a0 con 0x7fe7c4159990 2023-03-24T22:24:41.775 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7c2ffd700 1 --2- 172.21.15.18:0/555357804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7c406d200 0x7fe7c40c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:41.775 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7c37fe700 1 --2- 172.21.15.18:0/555357804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7c406b5b0 0x7fe7c41ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:41.775 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7c3fff700 1 --2- 172.21.15.18:0/555357804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c4159990 0x7fe7c40c9660 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00caa0 tx=0x7fe7bc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:41.775 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7abfff700 1 -- 172.21.15.18:0/555357804 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc003c30 con 0x7fe7c4159990 2023-03-24T22:24:41.775 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7abfff700 1 -- 172.21.15.18:0/555357804 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc003d90 con 0x7fe7c4159990 2023-03-24T22:24:41.776 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/555357804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c415d370 con 0x7fe7c4159990 2023-03-24T22:24:41.776 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7abfff700 1 -- 172.21.15.18:0/555357804 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc005270 con 0x7fe7c4159990 2023-03-24T22:24:41.776 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/555357804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c406c150 con 0x7fe7c4159990 2023-03-24T22:24:41.776 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7abfff700 1 -- 172.21.15.18:0/555357804 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe7bc0056d0 con 0x7fe7c4159990 2023-03-24T22:24:41.776 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.769+0000 7fe7abfff700 1 -- 172.21.15.18:0/555357804 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe7bc01cb10 con 0x7fe7c4159990 2023-03-24T22:24:41.777 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.770+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/555357804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c415d650 con 0x7fe7c4159990 2023-03-24T22:24:41.777 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.772+0000 7fe7abfff700 1 -- 172.21.15.18:0/555357804 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fe7bc021060 con 0x7fe7c4159990 2023-03-24T22:24:41.844 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.843+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/555357804 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7c404e850 con 0x7fe7c4159990 2023-03-24T22:24:41.845 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.844+0000 7fe7abfff700 1 -- 172.21.15.18:0/555357804 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe7bc017070 con 0x7fe7c4159990 2023-03-24T22:24:41.845 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:41.845 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":10,"fsid":"67a1ac4c-1568-462e-b9b7-d22a9a29a86a","created":"2023-03-24T22:24:39.069846+0000","modified":"2023-03-24T22:24:41.559093+0000","last_up_change":"0.000000","last_in_change":"2023-03-24T22:24:41.559093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d184f8d0-aaf7-421c-9c33-c9f32a0e9236","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d46c78b3-52ad-41f3-8c64-3d37e9adee63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4e4e783f-f720-42f9-b394-002a79f5eb0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2a8e5486-0ae6-43c3-870e-07ad88a43340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9a82df25-c818-47d7-8a9e-3e6fd55cd4f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ae37e2cd-d079-4962-9bd7-628aa5a7e14d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e3574199-2966-431b-ab0b-25efed52c749","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5a2a90ec-837f-4560-8a8e-b3e82eab590b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T22:24:41.846 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.845+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/555357804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c4159990 msgr2=0x7fe7c40c9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:41.846 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.845+0000 7fe7c9bf1700 1 --2- 172.21.15.18:0/555357804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c4159990 0x7fe7c40c9660 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00caa0 tx=0x7fe7bc00cdb0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.846 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.845+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/555357804 shutdown_connections 2023-03-24T22:24:41.847 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.845+0000 7fe7c9bf1700 1 --2- 172.21.15.18:0/555357804 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7c4159990 0x7fe7c40c9660 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.847 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.845+0000 7fe7c9bf1700 1 --2- 172.21.15.18:0/555357804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7c406b5b0 0x7fe7c41ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.847 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.845+0000 7fe7c9bf1700 1 --2- 172.21.15.18:0/555357804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7c406d200 0x7fe7c40c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:41.847 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.845+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/555357804 >> 172.21.15.18:0/555357804 conn(0x7fe7c40cd590 msgr2=0x7fe7c40cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:41.847 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.845+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/555357804 shutdown_connections 2023-03-24T22:24:41.847 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:41.845+0000 7fe7c9bf1700 1 -- 172.21.15.18:0/555357804 wait complete. 2023-03-24T22:24:41.858 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-03-24T22:24:41.858 INFO:tasks.ceph:Waiting for OSDs to come up 2023-03-24T22:24:41.902 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:41.901+0000 7f1e5e1b4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-24T22:24:41.903 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:41.903+0000 7f48f8982200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-24T22:24:42.192 INFO:tasks.ceph.osd.4.smithi100.stderr:2023-03-24T22:24:42.191+0000 7f580e5ab640 -1 Falling back to public interface 2023-03-24T22:24:42.193 INFO:tasks.ceph.osd.1.smithi018.stderr:2023-03-24T22:24:42.192+0000 7f25ebf10640 -1 Falling back to public interface 2023-03-24T22:24:42.196 INFO:tasks.ceph.osd.5.smithi100.stderr:2023-03-24T22:24:42.195+0000 7f5bc49ba640 -1 Falling back to public interface 2023-03-24T22:24:42.200 INFO:tasks.ceph.osd.7.smithi100.stderr:2023-03-24T22:24:42.199+0000 7f8468e04640 -1 Falling back to public interface 2023-03-24T22:24:42.207 INFO:tasks.ceph.osd.2.smithi018.stderr:2023-03-24T22:24:42.206+0000 7f59fbd48640 -1 Falling back to public interface 2023-03-24T22:24:42.217 INFO:tasks.ceph.osd.3.smithi018.stderr:2023-03-24T22:24:42.216+0000 7f972fc2d640 -1 Falling back to public interface 2023-03-24T22:24:42.218 INFO:tasks.ceph.osd.0.smithi018.stderr:2023-03-24T22:24:42.218+0000 7fd01906a640 -1 Falling back to public interface 2023-03-24T22:24:42.222 INFO:tasks.ceph.osd.6.smithi100.stderr:2023-03-24T22:24:42.221+0000 7fb8c576b640 -1 Falling back to public interface 2023-03-24T22:24:42.276 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:42.275+0000 7f1e5e1b4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-24T22:24:42.299 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:42.298+0000 7f48f8982200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-24T22:24:42.350 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:42.349+0000 7f1e5e1b4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-24T22:24:42.377 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:42.377+0000 7f48f8982200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-24T22:24:42.490 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:42.489+0000 7f1e5e1b4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-24T22:24:42.525 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:42.524+0000 7f48f8982200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-24T22:24:42.606 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:42.605+0000 7f1e5e1b4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-24T22:24:42.645 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:42.645+0000 7f48f8982200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-24T22:24:42.664 DEBUG:teuthology.orchestra.run.smithi018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-24T22:24:42.747 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:42.746+0000 7f1e5e1b4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-24T22:24:42.782 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.781+0000 7f3ea44df700 1 Processor -- start 2023-03-24T22:24:42.782 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.781+0000 7f3ea44df700 1 -- start start 2023-03-24T22:24:42.784 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.782+0000 7f3ea44df700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e9c158fc0 0x7f3e9c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:42.784 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.782+0000 7f3ea44df700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3e9c06e560 0x7f3e9c070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:42.784 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.782+0000 7f3ea44df700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c0cd900 con 0x7f3e9c070dc0 2023-03-24T22:24:42.784 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.782+0000 7f3ea44df700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c071200 con 0x7f3e9c06e560 2023-03-24T22:24:42.784 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.782+0000 7f3ea44df700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c06e070 con 0x7f3e9c158fc0 2023-03-24T22:24:42.785 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1a7a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3e9c06e560 0x7f3e9c070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:42.785 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1a7a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f3e9c06e560 0x7f3e9c070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56738/0 (socket says 172.21.15.18:56738) 2023-03-24T22:24:42.785 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1a7a700 1 -- 172.21.15.18:0/3815997807 learned_addr learned my addr 172.21.15.18:0/3815997807 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:42.785 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea227b700 1 --2- 172.21.15.18:0/3815997807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e9c158fc0 0x7f3e9c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:42.785 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1a7a700 1 -- 172.21.15.18:0/3815997807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e9c158fc0 msgr2=0x7f3e9c0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:42.785 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1a7a700 1 --2- 172.21.15.18:0/3815997807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e9c158fc0 0x7f3e9c0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:42.785 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1a7a700 1 -- 172.21.15.18:0/3815997807 >> v1:172.21.15.100:6789/0 conn(0x7f3e9c070dc0 legacy=0x7f3e9c06c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:42.786 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1a7a700 1 -- 172.21.15.18:0/3815997807 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c0aed30 con 0x7f3e9c06e560 2023-03-24T22:24:42.786 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1a7a700 1 --2- 172.21.15.18:0/3815997807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 0x7f3e9c070880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00bb90 tx=0x7f3e8c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9219ebccff2a71aa server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:42.786 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1279700 1 -- 172.21.15.18:0/3815997807 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c00dd30 con 0x7f3e9c06e560 2023-03-24T22:24:42.786 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1279700 1 -- 172.21.15.18:0/3815997807 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c00de90 con 0x7f3e9c06e560 2023-03-24T22:24:42.786 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.783+0000 7f3ea1279700 1 -- 172.21.15.18:0/3815997807 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c010690 con 0x7f3e9c06e560 2023-03-24T22:24:42.788 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.784+0000 7f3ea44df700 1 -- 172.21.15.18:0/3815997807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 msgr2=0x7f3e9c070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:42.788 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.784+0000 7f3ea44df700 1 --2- 172.21.15.18:0/3815997807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 0x7f3e9c070880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00bb90 tx=0x7f3e8c00bea0 comp rx=0 tx=0).stop 2023-03-24T22:24:42.788 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.784+0000 7f3ea44df700 1 -- 172.21.15.18:0/3815997807 shutdown_connections 2023-03-24T22:24:42.788 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.784+0000 7f3ea44df700 1 --2- 172.21.15.18:0/3815997807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e9c158fc0 0x7f3e9c0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:42.789 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.784+0000 7f3ea44df700 1 --2- 172.21.15.18:0/3815997807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 0x7f3e9c070880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:42.789 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.784+0000 7f3ea44df700 1 -- 172.21.15.18:0/3815997807 >> 172.21.15.18:0/3815997807 conn(0x7f3e9c0ccd80 msgr2=0x7f3e9c0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:42.789 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.784+0000 7f3ea44df700 1 -- 172.21.15.18:0/3815997807 shutdown_connections 2023-03-24T22:24:42.789 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.784+0000 7f3ea44df700 1 -- 172.21.15.18:0/3815997807 wait complete. 2023-03-24T22:24:42.789 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.785+0000 7f3ea44df700 1 Processor -- start 2023-03-24T22:24:42.789 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.785+0000 7f3ea44df700 1 -- start start 2023-03-24T22:24:42.790 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea44df700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 0x7f3e9c1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:42.790 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea44df700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9c070dc0 0x7f3e9c155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:42.790 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea44df700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e9c158fc0 0x7f3e9c151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:42.790 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea44df700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c152220 con 0x7f3e9c06e560 2023-03-24T22:24:42.790 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea44df700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c1524f0 con 0x7f3e9c070dc0 2023-03-24T22:24:42.790 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea44df700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c1527d0 con 0x7f3e9c158fc0 2023-03-24T22:24:42.790 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea227b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 0x7f3e9c1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:42.791 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea227b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 0x7f3e9c1516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56744/0 (socket says 172.21.15.18:56744) 2023-03-24T22:24:42.791 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea227b700 1 -- 172.21.15.18:0/878176166 learned_addr learned my addr 172.21.15.18:0/878176166 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:42.791 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea2a7c700 1 --2- 172.21.15.18:0/878176166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e9c158fc0 0x7f3e9c151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:42.791 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea1a7a700 1 --2- 172.21.15.18:0/878176166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9c070dc0 0x7f3e9c155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:42.791 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea227b700 1 -- 172.21.15.18:0/878176166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e9c158fc0 msgr2=0x7f3e9c151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:42.791 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea227b700 1 --2- 172.21.15.18:0/878176166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e9c158fc0 0x7f3e9c151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:42.791 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea227b700 1 -- 172.21.15.18:0/878176166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9c070dc0 msgr2=0x7f3e9c155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:42.791 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea227b700 1 --2- 172.21.15.18:0/878176166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9c070dc0 0x7f3e9c155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:42.792 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.786+0000 7f3ea227b700 1 -- 172.21.15.18:0/878176166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c00b7f0 con 0x7f3e9c06e560 2023-03-24T22:24:42.792 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.787+0000 7f3ea227b700 1 --2- 172.21.15.18:0/878176166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 0x7f3e9c1516e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e870 tx=0x7f3e9800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:42.792 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.787+0000 7f3e92ffd700 1 -- 172.21.15.18:0/878176166 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98009590 con 0x7f3e9c06e560 2023-03-24T22:24:42.792 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.787+0000 7f3e92ffd700 1 -- 172.21.15.18:0/878176166 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e98013680 con 0x7f3e9c06e560 2023-03-24T22:24:42.792 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.787+0000 7f3ea44df700 1 -- 172.21.15.18:0/878176166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c06b030 con 0x7f3e9c06e560 2023-03-24T22:24:42.792 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.787+0000 7f3e92ffd700 1 -- 172.21.15.18:0/878176166 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e980139f0 con 0x7f3e9c06e560 2023-03-24T22:24:42.792 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.787+0000 7f3ea44df700 1 -- 172.21.15.18:0/878176166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c0cd620 con 0x7f3e9c06e560 2023-03-24T22:24:42.793 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.787+0000 7f3e92ffd700 1 -- 172.21.15.18:0/878176166 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3e98005a20 con 0x7f3e9c06e560 2023-03-24T22:24:42.793 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.787+0000 7f3e92ffd700 1 -- 172.21.15.18:0/878176166 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f3e98006780 con 0x7f3e9c06e560 2023-03-24T22:24:42.793 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.788+0000 7f3ea44df700 1 -- 172.21.15.18:0/878176166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c06b310 con 0x7f3e9c06e560 2023-03-24T22:24:42.794 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.790+0000 7f3e92ffd700 1 -- 172.21.15.18:0/878176166 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f3e98021060 con 0x7f3e9c06e560 2023-03-24T22:24:42.796 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:42.796+0000 7f48f8982200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-24T22:24:42.862 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.861+0000 7f3ea44df700 1 -- 172.21.15.18:0/878176166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e9c0c3340 con 0x7f3e9c06e560 2023-03-24T22:24:42.862 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.861+0000 7f3e92ffd700 1 -- 172.21.15.18:0/878176166 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f3e98017070 con 0x7f3e9c06e560 2023-03-24T22:24:42.863 INFO:teuthology.misc.health.smithi018.stdout: 2023-03-24T22:24:42.863 INFO:teuthology.misc.health.smithi018.stdout:{"epoch":10,"fsid":"67a1ac4c-1568-462e-b9b7-d22a9a29a86a","created":"2023-03-24T22:24:39.069846+0000","modified":"2023-03-24T22:24:41.559093+0000","last_up_change":"0.000000","last_in_change":"2023-03-24T22:24:41.559093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d184f8d0-aaf7-421c-9c33-c9f32a0e9236","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d46c78b3-52ad-41f3-8c64-3d37e9adee63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4e4e783f-f720-42f9-b394-002a79f5eb0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2a8e5486-0ae6-43c3-870e-07ad88a43340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9a82df25-c818-47d7-8a9e-3e6fd55cd4f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ae37e2cd-d079-4962-9bd7-628aa5a7e14d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e3574199-2966-431b-ab0b-25efed52c749","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5a2a90ec-837f-4560-8a8e-b3e82eab590b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T22:24:42.864 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.862+0000 7f3ea44df700 1 -- 172.21.15.18:0/878176166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 msgr2=0x7f3e9c1516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:42.864 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.862+0000 7f3ea44df700 1 --2- 172.21.15.18:0/878176166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 0x7f3e9c1516e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e870 tx=0x7f3e9800eb80 comp rx=0 tx=0).stop 2023-03-24T22:24:42.864 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.862+0000 7f3ea44df700 1 -- 172.21.15.18:0/878176166 shutdown_connections 2023-03-24T22:24:42.864 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.862+0000 7f3ea44df700 1 --2- 172.21.15.18:0/878176166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e9c06e560 0x7f3e9c1516e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:42.864 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.862+0000 7f3ea44df700 1 --2- 172.21.15.18:0/878176166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e9c158fc0 0x7f3e9c151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:42.864 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.862+0000 7f3ea44df700 1 --2- 172.21.15.18:0/878176166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e9c070dc0 0x7f3e9c155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:42.864 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.862+0000 7f3ea44df700 1 -- 172.21.15.18:0/878176166 >> 172.21.15.18:0/878176166 conn(0x7f3e9c0ccd80 msgr2=0x7f3e9c0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:42.865 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.862+0000 7f3ea44df700 1 -- 172.21.15.18:0/878176166 shutdown_connections 2023-03-24T22:24:42.865 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:42.862+0000 7f3ea44df700 1 -- 172.21.15.18:0/878176166 wait complete. 2023-03-24T22:24:42.875 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-03-24T22:24:42.963 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:42.962+0000 7f1e5e1b4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-24T22:24:43.022 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:43.022+0000 7f48f8982200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-24T22:24:43.044 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:43.043+0000 7f1e5e1b4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-24T22:24:43.107 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:43.106+0000 7f48f8982200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-24T22:24:43.122 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:43.121+0000 7f1e5e1b4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-24T22:24:43.188 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:43.187+0000 7f48f8982200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-24T22:24:43.910 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:43.909+0000 7f1e5e1b4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-24T22:24:43.993 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:43.992+0000 7f1e5e1b4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-24T22:24:44.017 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:44.016+0000 7f48f8982200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-24T22:24:44.069 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:44.067+0000 7f1e5e1b4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-24T22:24:44.104 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:44.103+0000 7f48f8982200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-24T22:24:44.152 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:44.151+0000 7f1e5e1b4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-24T22:24:44.184 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:44.183+0000 7f48f8982200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-24T22:24:44.271 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:44.270+0000 7f48f8982200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-24T22:24:44.303 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:44.302+0000 7f1e5e1b4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-03-24T22:24:44.427 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:44.426+0000 7f48f8982200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-03-24T22:24:44.486 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:44.485+0000 7f1e5e1b4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-24T22:24:44.559 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:44.558+0000 7f1e5e1b4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-24T22:24:44.620 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:44.619+0000 7f48f8982200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-24T22:24:44.710 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:44.709+0000 7f48f8982200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-24T22:24:44.721 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:44.720+0000 7f1e5e1b4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-24T22:24:44.875 INFO:tasks.ceph.mgr.y.smithi018.stderr:2023-03-24T22:24:44.874+0000 7f48f8982200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-24T22:24:45.694 INFO:tasks.ceph.osd.1.smithi018.stderr:2023-03-24T22:24:45.693+0000 7f25ebf10640 -1 osd.1 0 log_to_monitors true 2023-03-24T22:24:45.704 INFO:tasks.ceph.osd.2.smithi018.stderr:2023-03-24T22:24:45.703+0000 7f59fbd48640 -1 osd.2 0 log_to_monitors true 2023-03-24T22:24:45.721 INFO:tasks.ceph.osd.7.smithi100.stderr:2023-03-24T22:24:45.720+0000 7f8468e04640 -1 osd.7 0 log_to_monitors true 2023-03-24T22:24:45.722 INFO:tasks.ceph.osd.4.smithi100.stderr:2023-03-24T22:24:45.721+0000 7f580e5ab640 -1 osd.4 0 log_to_monitors true 2023-03-24T22:24:45.723 INFO:tasks.ceph.osd.3.smithi018.stderr:2023-03-24T22:24:45.722+0000 7f972fc2d640 -1 osd.3 0 log_to_monitors true 2023-03-24T22:24:45.741 INFO:tasks.ceph.osd.0.smithi018.stderr:2023-03-24T22:24:45.740+0000 7fd01906a640 -1 osd.0 0 log_to_monitors true 2023-03-24T22:24:45.747 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:45.746+0000 7f1e17c90700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-03-24T22:24:45.748 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-03-24T22:24:45.747+0000 7f1e17c90700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-03-24T22:24:45.751 INFO:tasks.ceph.osd.5.smithi100.stderr:2023-03-24T22:24:45.750+0000 7f5bc49ba640 -1 osd.5 0 log_to_monitors true 2023-03-24T22:24:45.775 INFO:tasks.ceph.osd.6.smithi100.stderr:2023-03-24T22:24:45.774+0000 7fb8c576b640 -1 osd.6 0 log_to_monitors true 2023-03-24T22:24:48.271 INFO:tasks.ceph.osd.3.smithi018.stderr:2023-03-24T22:24:48.270+0000 7f97237d3700 -1 osd.3 0 waiting for initial osdmap 2023-03-24T22:24:48.282 INFO:tasks.ceph.osd.3.smithi018.stderr:2023-03-24T22:24:48.282+0000 7f971bbae700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T22:24:48.288 INFO:tasks.ceph.osd.2.smithi018.stderr:2023-03-24T22:24:48.287+0000 7f59ef8ee700 -1 osd.2 0 waiting for initial osdmap 2023-03-24T22:24:48.292 INFO:tasks.ceph.osd.0.smithi018.stderr:2023-03-24T22:24:48.292+0000 7fd00cc10700 -1 osd.0 0 waiting for initial osdmap 2023-03-24T22:24:48.301 INFO:tasks.ceph.osd.2.smithi018.stderr:2023-03-24T22:24:48.301+0000 7f59e7cc9700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T22:24:48.305 INFO:tasks.ceph.osd.0.smithi018.stderr:2023-03-24T22:24:48.305+0000 7fd004feb700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T22:24:48.312 INFO:tasks.ceph.osd.1.smithi018.stderr:2023-03-24T22:24:48.311+0000 7f25dfab6700 -1 osd.1 0 waiting for initial osdmap 2023-03-24T22:24:48.326 INFO:tasks.ceph.osd.1.smithi018.stderr:2023-03-24T22:24:48.325+0000 7f25d7e91700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T22:24:48.881 INFO:tasks.ceph.osd.4.smithi100.stderr:2023-03-24T22:24:48.880+0000 7f5802151700 -1 osd.4 0 waiting for initial osdmap 2023-03-24T22:24:48.895 INFO:tasks.ceph.osd.4.smithi100.stderr:2023-03-24T22:24:48.894+0000 7f57fa52c700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T22:24:48.897 INFO:tasks.ceph.osd.7.smithi100.stderr:2023-03-24T22:24:48.896+0000 7f845c9aa700 -1 osd.7 0 waiting for initial osdmap 2023-03-24T22:24:48.903 INFO:tasks.ceph.osd.5.smithi100.stderr:2023-03-24T22:24:48.902+0000 7f5bb8560700 -1 osd.5 0 waiting for initial osdmap 2023-03-24T22:24:48.909 INFO:tasks.ceph.osd.7.smithi100.stderr:2023-03-24T22:24:48.908+0000 7f8454d85700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T22:24:48.910 INFO:tasks.ceph.osd.6.smithi100.stderr:2023-03-24T22:24:48.909+0000 7fb8b9311700 -1 osd.6 0 waiting for initial osdmap 2023-03-24T22:24:48.921 INFO:tasks.ceph.osd.5.smithi100.stderr:2023-03-24T22:24:48.920+0000 7f5bb093b700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T22:24:48.930 INFO:tasks.ceph.osd.6.smithi100.stderr:2023-03-24T22:24:48.929+0000 7fb8b16ec700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-24T22:24:49.681 DEBUG:teuthology.orchestra.run.smithi018:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-24T22:24:49.812 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.811+0000 7ffb87b03700 1 Processor -- start 2023-03-24T22:24:49.812 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.811+0000 7ffb87b03700 1 -- start start 2023-03-24T22:24:49.815 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb87b03700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffb8015a6e0 0x7ffb800ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:49.815 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb87b03700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb80071130 0x7ffb800715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:49.815 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb87b03700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8015cf60 con 0x7ffb80160530 2023-03-24T22:24:49.815 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb87b03700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb80071af0 con 0x7ffb8015a6e0 2023-03-24T22:24:49.815 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb87b03700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb801609a0 con 0x7ffb80071130 2023-03-24T22:24:49.815 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8509e700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7ffb80160530 0x7ffb8006e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38158/0 (socket says 172.21.15.18:38158) 2023-03-24T22:24:49.816 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8509e700 1 -- 172.21.15.18:0/408112636 learned_addr learned my addr 172.21.15.18:0/408112636 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:49.816 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb860a0700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb80071130 0x7ffb800715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:49.816 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8589f700 1 --2- 172.21.15.18:0/408112636 >> v2:172.21.15.100:3300/0 conn(0x7ffb8015a6e0 0x7ffb800ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:49.816 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089444419 0 0) 0x7ffb8015cf60 con 0x7ffb80160530 2023-03-24T22:24:49.816 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb6c0035b0 con 0x7ffb80160530 2023-03-24T22:24:49.816 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 <== mon.0 v1:172.21.15.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1086199687 0 0) 0x7ffb6c0035b0 con 0x7ffb80160530 2023-03-24T22:24:49.816 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb80071130 msgr2=0x7ffb800715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:49.817 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8489d700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb80071130 0x7ffb800715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:49.817 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 >> v2:172.21.15.100:3300/0 conn(0x7ffb8015a6e0 msgr2=0x7ffb800ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:49.817 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8489d700 1 --2- 172.21.15.18:0/408112636 >> v2:172.21.15.100:3300/0 conn(0x7ffb8015a6e0 0x7ffb800ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:49.817 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.812+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 --> v1:172.21.15.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb80160c80 con 0x7ffb80160530 2023-03-24T22:24:49.817 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 <== mon.0 v1:172.21.15.18:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3658844337 0 0) 0x7ffb70001670 con 0x7ffb80160530 2023-03-24T22:24:49.817 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 >> v1:172.21.15.18:6789/0 conn(0x7ffb80160530 legacy=0x7ffb8006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:49.817 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb6c003a00 0x7ffb6c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:49.818 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6c004430 0x7ffb6c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:49.818 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb80071130 0x7ffb6c004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:49.818 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb80071af0 con 0x7ffb6c003a00 2023-03-24T22:24:49.818 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb801609a0 con 0x7ffb6c004430 2023-03-24T22:24:49.818 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c00ba10 con 0x7ffb80071130 2023-03-24T22:24:49.818 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8589f700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb6c003a00 0x7ffb6c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:49.818 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8509e700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6c004430 0x7ffb6c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:49.818 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb860a0700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb80071130 0x7ffb6c004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:49.819 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8589f700 1 -- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb80071130 msgr2=0x7ffb6c004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:49.819 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8589f700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb80071130 0x7ffb6c004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:49.819 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8589f700 1 -- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6c004430 msgr2=0x7ffb6c0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:49.819 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8589f700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6c004430 0x7ffb6c0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:49.819 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8589f700 1 -- 172.21.15.18:0/408112636 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c0035b0 con 0x7ffb6c003a00 2023-03-24T22:24:49.819 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb860a0700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb80071130 0x7ffb6c004fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-24T22:24:49.819 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8589f700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb6c003a00 0x7ffb6c003e10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009b60 tx=0x7ffb6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabfbafaf81e407c server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:49.820 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00e6c0 con 0x7ffb6c003a00 2023-03-24T22:24:49.820 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c00e820 con 0x7ffb6c003a00 2023-03-24T22:24:49.820 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.813+0000 7ffb8489d700 1 -- 172.21.15.18:0/408112636 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00eb90 con 0x7ffb6c003a00 2023-03-24T22:24:49.820 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.814+0000 7ffb87b03700 1 -- 172.21.15.18:0/408112636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb6c003a00 msgr2=0x7ffb6c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:49.820 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.814+0000 7ffb87b03700 1 --2- 172.21.15.18:0/408112636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb6c003a00 0x7ffb6c003e10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009b60 tx=0x7ffb6c005880 comp rx=0 tx=0).stop 2023-03-24T22:24:49.820 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.814+0000 7ffb860a0700 1 -- 172.21.15.18:0/408112636 reap_dead start 2023-03-24T22:24:49.820 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.814+0000 7ffb87b03700 1 -- 172.21.15.18:0/408112636 shutdown_connections 2023-03-24T22:24:49.820 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.814+0000 7ffb87b03700 1 -- 172.21.15.18:0/408112636 >> 172.21.15.18:0/408112636 conn(0x7ffb800cd590 msgr2=0x7ffb800cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:49.821 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.814+0000 7ffb87b03700 1 -- 172.21.15.18:0/408112636 shutdown_connections 2023-03-24T22:24:49.821 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.814+0000 7ffb87b03700 1 -- 172.21.15.18:0/408112636 wait complete. 2023-03-24T22:24:49.823 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.815+0000 7ffb87b03700 1 Processor -- start 2023-03-24T22:24:49.823 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.815+0000 7ffb87b03700 1 -- start start 2023-03-24T22:24:49.823 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb87b03700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6c003a00 0x7ffb800c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:49.823 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb87b03700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb80154f60 0x7ffb801553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:49.823 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb87b03700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb800c7210 0x7ffb800c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:49.824 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb87b03700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb80155910 con 0x7ffb80154f60 2023-03-24T22:24:49.824 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb87b03700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800ae2e0 con 0x7ffb6c003a00 2023-03-24T22:24:49.824 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb87b03700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800ae5c0 con 0x7ffb800c7210 2023-03-24T22:24:49.824 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb8509e700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb80154f60 0x7ffb801553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:49.824 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb8509e700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb80154f60 0x7ffb801553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43820/0 (socket says 172.21.15.18:43820) 2023-03-24T22:24:49.824 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb8509e700 1 -- 172.21.15.18:0/2707098763 learned_addr learned my addr 172.21.15.18:0/2707098763 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:49.824 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb860a0700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb800c7210 0x7ffb800c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:49.825 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.816+0000 7ffb8589f700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6c003a00 0x7ffb800c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:49.825 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb8509e700 1 -- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb800c7210 msgr2=0x7ffb800c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:49.825 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb8509e700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb800c7210 0x7ffb800c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:49.825 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb8509e700 1 -- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6c003a00 msgr2=0x7ffb800c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:49.825 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb8509e700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6c003a00 0x7ffb800c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:49.825 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb8509e700 1 -- 172.21.15.18:0/2707098763 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb80161240 con 0x7ffb80154f60 2023-03-24T22:24:49.825 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb8509e700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb80154f60 0x7ffb801553d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffb70000fb0 tx=0x7ffb700075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:49.826 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb767fc700 1 -- 172.21.15.18:0/2707098763 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb700077d0 con 0x7ffb80154f60 2023-03-24T22:24:49.826 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb767fc700 1 -- 172.21.15.18:0/2707098763 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7000f8a0 con 0x7ffb80154f60 2023-03-24T22:24:49.826 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb767fc700 1 -- 172.21.15.18:0/2707098763 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000fc50 con 0x7ffb80154f60 2023-03-24T22:24:49.826 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb87b03700 1 -- 172.21.15.18:0/2707098763 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb80160f60 con 0x7ffb80154f60 2023-03-24T22:24:49.826 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.817+0000 7ffb87b03700 1 -- 172.21.15.18:0/2707098763 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8006f090 con 0x7ffb80154f60 2023-03-24T22:24:49.826 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.818+0000 7ffb87b03700 1 -- 172.21.15.18:0/2707098763 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8004e850 con 0x7ffb80154f60 2023-03-24T22:24:49.826 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.818+0000 7ffb767fc700 1 -- 172.21.15.18:0/2707098763 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 87553+0+0 (secure 0 0 0) 0x7ffb7001d460 con 0x7ffb80154f60 2023-03-24T22:24:49.826 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.819+0000 7ffb767fc700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7ffb64069d50 0x7ffb6406c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:49.827 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.819+0000 7ffb767fc700 1 -- 172.21.15.18:0/2707098763 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7ffb70087980 con 0x7ffb80154f60 2023-03-24T22:24:49.827 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.819+0000 7ffb8589f700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7ffb64069d50 0x7ffb6406c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:49.827 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.819+0000 7ffb8589f700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7ffb64069d50 0x7ffb6406c200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0090b0 tx=0x7ffb7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:49.827 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:49.823+0000 7ffb767fc700 1 -- 172.21.15.18:0/2707098763 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7ffb70053840 con 0x7ffb80154f60 2023-03-24T22:24:50.022 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.021+0000 7ffb87b03700 1 -- 172.21.15.18:0/2707098763 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb800d0b30 con 0x7ffb80154f60 2023-03-24T22:24:50.022 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.021+0000 7ffb767fc700 1 -- 172.21.15.18:0/2707098763 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10536 (secure 0 0 0) 0x7ffb70056e60 con 0x7ffb80154f60 2023-03-24T22:24:50.023 INFO:teuthology.misc.health.smithi018.stdout: 2023-03-24T22:24:50.023 INFO:teuthology.misc.health.smithi018.stdout:{"epoch":13,"fsid":"67a1ac4c-1568-462e-b9b7-d22a9a29a86a","created":"2023-03-24T22:24:39.069846+0000","modified":"2023-03-24T22:24:49.299255+0000","last_up_change":"2023-03-24T22:24:49.299255+0000","last_in_change":"2023-03-24T22:24:41.559093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d184f8d0-aaf7-421c-9c33-c9f32a0e9236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":114916},{"type":"v1","addr":"172.21.15.18:6821","nonce":114916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":114916},{"type":"v1","addr":"172.21.15.18:6825","nonce":114916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":114916},{"type":"v1","addr":"172.21.15.18:6831","nonce":114916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6827","nonce":114916},{"type":"v1","addr":"172.21.15.18:6829","nonce":114916}]},"public_addr":"172.21.15.18:6821/114916","cluster_addr":"172.21.15.18:6825/114916","heartbeat_back_addr":"172.21.15.18:6831/114916","heartbeat_front_addr":"172.21.15.18:6829/114916","state":["exists","up"]},{"osd":1,"uuid":"d46c78b3-52ad-41f3-8c64-3d37e9adee63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":114913},{"type":"v1","addr":"172.21.15.18:6801","nonce":114913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":114913},{"type":"v1","addr":"172.21.15.18:6803","nonce":114913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":114913},{"type":"v1","addr":"172.21.15.18:6807","nonce":114913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":114913},{"type":"v1","addr":"172.21.15.18:6805","nonce":114913}]},"public_addr":"172.21.15.18:6801/114913","cluster_addr":"172.21.15.18:6803/114913","heartbeat_back_addr":"172.21.15.18:6807/114913","heartbeat_front_addr":"172.21.15.18:6805/114913","state":["exists","up"]},{"osd":2,"uuid":"4e4e783f-f720-42f9-b394-002a79f5eb0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":114914},{"type":"v1","addr":"172.21.15.18:6809","nonce":114914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":114914},{"type":"v1","addr":"172.21.15.18:6811","nonce":114914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":114914},{"type":"v1","addr":"172.21.15.18:6815","nonce":114914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":114914},{"type":"v1","addr":"172.21.15.18:6813","nonce":114914}]},"public_addr":"172.21.15.18:6809/114914","cluster_addr":"172.21.15.18:6811/114914","heartbeat_back_addr":"172.21.15.18:6815/114914","heartbeat_front_addr":"172.21.15.18:6813/114914","state":["exists","up"]},{"osd":3,"uuid":"2a8e5486-0ae6-43c3-870e-07ad88a43340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":114915},{"type":"v1","addr":"172.21.15.18:6817","nonce":114915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":114915},{"type":"v1","addr":"172.21.15.18:6820","nonce":114915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":114915},{"type":"v1","addr":"172.21.15.18:6828","nonce":114915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":114915},{"type":"v1","addr":"172.21.15.18:6824","nonce":114915}]},"public_addr":"172.21.15.18:6817/114915","cluster_addr":"172.21.15.18:6820/114915","heartbeat_back_addr":"172.21.15.18:6828/114915","heartbeat_front_addr":"172.21.15.18:6824/114915","state":["exists","up"]},{"osd":4,"uuid":"9a82df25-c818-47d7-8a9e-3e6fd55cd4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":115214},{"type":"v1","addr":"172.21.15.100:6801","nonce":115214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":115214},{"type":"v1","addr":"172.21.15.100:6803","nonce":115214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":115214},{"type":"v1","addr":"172.21.15.100:6807","nonce":115214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":115214},{"type":"v1","addr":"172.21.15.100:6805","nonce":115214}]},"public_addr":"172.21.15.100:6801/115214","cluster_addr":"172.21.15.100:6803/115214","heartbeat_back_addr":"172.21.15.100:6807/115214","heartbeat_front_addr":"172.21.15.100:6805/115214","state":["exists","up"]},{"osd":5,"uuid":"ae37e2cd-d079-4962-9bd7-628aa5a7e14d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":115215},{"type":"v1","addr":"172.21.15.100:6809","nonce":115215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":115215},{"type":"v1","addr":"172.21.15.100:6811","nonce":115215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":115215},{"type":"v1","addr":"172.21.15.100:6815","nonce":115215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":115215},{"type":"v1","addr":"172.21.15.100:6813","nonce":115215}]},"public_addr":"172.21.15.100:6809/115215","cluster_addr":"172.21.15.100:6811/115215","heartbeat_back_addr":"172.21.15.100:6815/115215","heartbeat_front_addr":"172.21.15.100:6813/115215","state":["exists","up"]},{"osd":6,"uuid":"e3574199-2966-431b-ab0b-25efed52c749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":115255},{"type":"v1","addr":"172.21.15.100:6825","nonce":115255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":115255},{"type":"v1","addr":"172.21.15.100:6827","nonce":115255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":115255},{"type":"v1","addr":"172.21.15.100:6831","nonce":115255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":115255},{"type":"v1","addr":"172.21.15.100:6829","nonce":115255}]},"public_addr":"172.21.15.100:6825/115255","cluster_addr":"172.21.15.100:6827/115255","heartbeat_back_addr":"172.21.15.100:6831/115255","heartbeat_front_addr":"172.21.15.100:6829/115255","state":["exists","up"]},{"osd":7,"uuid":"5a2a90ec-837f-4560-8a8e-b3e82eab590b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":115216},{"type":"v1","addr":"172.21.15.100:6817","nonce":115216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":115216},{"type":"v1","addr":"172.21.15.100:6819","nonce":115216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":115216},{"type":"v1","addr":"172.21.15.100:6823","nonce":115216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":115216},{"type":"v1","addr":"172.21.15.100:6821","nonce":115216}]},"public_addr":"172.21.15.100:6817/115216","cluster_addr":"172.21.15.100:6819/115216","heartbeat_back_addr":"172.21.15.100:6823/115216","heartbeat_front_addr":"172.21.15.100:6821/115216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T22:24:50.025 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 -- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7ffb64069d50 msgr2=0x7ffb6406c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:50.025 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7ffb64069d50 0x7ffb6406c200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0090b0 tx=0x7ffb7c008030 comp rx=0 tx=0).stop 2023-03-24T22:24:50.025 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 -- 172.21.15.18:0/2707098763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb80154f60 msgr2=0x7ffb801553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:50.025 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb80154f60 0x7ffb801553d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffb70000fb0 tx=0x7ffb700075f0 comp rx=0 tx=0).stop 2023-03-24T22:24:50.025 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 -- 172.21.15.18:0/2707098763 shutdown_connections 2023-03-24T22:24:50.025 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7ffb64069d50 0x7ffb6406c200 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:50.026 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb800c7210 0x7ffb800c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:50.026 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb6c003a00 0x7ffb800c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:50.026 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 --2- 172.21.15.18:0/2707098763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb80154f60 0x7ffb801553d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:50.026 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 -- 172.21.15.18:0/2707098763 >> 172.21.15.18:0/2707098763 conn(0x7ffb800cd590 msgr2=0x7ffb8015d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:50.026 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 -- 172.21.15.18:0/2707098763 shutdown_connections 2023-03-24T22:24:50.026 INFO:teuthology.misc.health.smithi018.stderr:2023-03-24T22:24:50.024+0000 7ffb87b03700 1 -- 172.21.15.18:0/2707098763 wait complete. 2023-03-24T22:24:50.036 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-03-24T22:24:50.036 INFO:tasks.ceph:Creating RBD pool 2023-03-24T22:24:50.036 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd pool create rbd 8 2023-03-24T22:24:50.162 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.161+0000 7f185adf8700 1 Processor -- start 2023-03-24T22:24:50.162 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.161+0000 7f185adf8700 1 -- start start 2023-03-24T22:24:50.163 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f185adf8700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f18540b5f80 0x7f18540cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:50.164 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f185adf8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18540d2830 con 0x7f18540d0010 2023-03-24T22:24:50.164 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f185adf8700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18540d2df0 con 0x7f18540b23c0 2023-03-24T22:24:50.164 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f185adf8700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18540b2790 con 0x7f18540b5f80 2023-03-24T22:24:50.164 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f1853fff700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f18540b5f80 0x7f18540cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:50.164 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f1859395700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f18540d0010 0x7f18540d3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38170/0 (socket says 172.21.15.18:38170) 2023-03-24T22:24:50.164 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f1859395700 1 -- 172.21.15.18:0/2403277405 learned_addr learned my addr 172.21.15.18:0/2403277405 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:50.164 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f1853fff700 1 -- 172.21.15.18:0/2403277405 >> v1:172.21.15.100:6789/0 conn(0x7f18540b23c0 legacy=0x7f18540d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:50.165 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f1853fff700 1 -- 172.21.15.18:0/2403277405 >> v1:172.21.15.18:6789/0 conn(0x7f18540d0010 legacy=0x7f18540d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:50.165 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f18537fe700 1 -- 172.21.15.18:0/2403277405 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263820680 0 0) 0x7f18540d2830 con 0x7f18540d0010 2023-03-24T22:24:50.165 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f1853fff700 1 -- 172.21.15.18:0/2403277405 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540b6400 con 0x7f18540b5f80 2023-03-24T22:24:50.165 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.162+0000 7f1853fff700 1 --2- 172.21.15.18:0/2403277405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540b5f80 0x7f18540cfad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1840009840 tx=0x7f1840009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd3123ee15a45ed server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:50.165 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.163+0000 7f18537fe700 1 -- 172.21.15.18:0/2403277405 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840011830 con 0x7f18540b5f80 2023-03-24T22:24:50.165 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.163+0000 7f18537fe700 1 -- 172.21.15.18:0/2403277405 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1840011990 con 0x7f18540b5f80 2023-03-24T22:24:50.165 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.163+0000 7f18537fe700 1 -- 172.21.15.18:0/2403277405 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840011d20 con 0x7f18540b5f80 2023-03-24T22:24:50.167 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.163+0000 7f185adf8700 1 -- 172.21.15.18:0/2403277405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540b5f80 msgr2=0x7f18540cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:50.168 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.163+0000 7f185adf8700 1 --2- 172.21.15.18:0/2403277405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540b5f80 0x7f18540cfad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1840009840 tx=0x7f1840009b50 comp rx=0 tx=0).stop 2023-03-24T22:24:50.168 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.163+0000 7f185adf8700 1 -- 172.21.15.18:0/2403277405 shutdown_connections 2023-03-24T22:24:50.168 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.163+0000 7f185adf8700 1 --2- 172.21.15.18:0/2403277405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540b5f80 0x7f18540cfad0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:50.168 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.163+0000 7f185adf8700 1 -- 172.21.15.18:0/2403277405 >> 172.21.15.18:0/2403277405 conn(0x7f1854066140 msgr2=0x7f1854066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:50.168 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.163+0000 7f185adf8700 1 -- 172.21.15.18:0/2403277405 shutdown_connections 2023-03-24T22:24:50.168 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.163+0000 7f185adf8700 1 -- 172.21.15.18:0/2403277405 wait complete. 2023-03-24T22:24:50.169 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.164+0000 7f185adf8700 1 Processor -- start 2023-03-24T22:24:50.169 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.164+0000 7f185adf8700 1 -- start start 2023-03-24T22:24:50.169 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.165+0000 7f185adf8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18540b23c0 0x7f185415f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:50.169 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.165+0000 7f185adf8700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18540b5f80 0x7f18541518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:50.169 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.165+0000 7f185adf8700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540d0010 0x7f18540c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:50.169 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.165+0000 7f185adf8700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185415d0e0 con 0x7f18540d0010 2023-03-24T22:24:50.169 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.165+0000 7f185adf8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1854151e10 con 0x7f18540b23c0 2023-03-24T22:24:50.170 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.165+0000 7f185adf8700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18540c9170 con 0x7f18540b5f80 2023-03-24T22:24:50.170 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.165+0000 7f1859395700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540d0010 0x7f18540c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:50.170 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.165+0000 7f1859395700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540d0010 0x7f18540c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43842/0 (socket says 172.21.15.18:43842) 2023-03-24T22:24:50.170 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.165+0000 7f1859395700 1 -- 172.21.15.18:0/897342608 learned_addr learned my addr 172.21.15.18:0/897342608 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:50.170 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f1858b94700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18540b23c0 0x7f185415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:50.170 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f1853fff700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18540b5f80 0x7f18541518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:50.170 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f1859395700 1 -- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18540b5f80 msgr2=0x7f18541518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:50.170 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f1859395700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18540b5f80 0x7f18541518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:50.171 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f1859395700 1 -- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18540b23c0 msgr2=0x7f185415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:50.171 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f1859395700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18540b23c0 0x7f185415f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:50.171 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f1859395700 1 -- 172.21.15.18:0/897342608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18400094a0 con 0x7f18540d0010 2023-03-24T22:24:50.171 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f1853fff700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18540b5f80 0x7f18541518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-24T22:24:50.171 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f1859395700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540d0010 0x7f18540c8c30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f184c00cc70 tx=0x7f184c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:50.171 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f18517fa700 1 -- 172.21.15.18:0/897342608 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c00ada0 con 0x7f18540d0010 2023-03-24T22:24:50.171 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f18517fa700 1 -- 172.21.15.18:0/897342608 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c00ce70 con 0x7f18540d0010 2023-03-24T22:24:50.171 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f185adf8700 1 -- 172.21.15.18:0/897342608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18540cdc00 con 0x7f18540d0010 2023-03-24T22:24:50.172 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f18517fa700 1 -- 172.21.15.18:0/897342608 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c0072f0 con 0x7f18540d0010 2023-03-24T22:24:50.172 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.166+0000 7f185adf8700 1 -- 172.21.15.18:0/897342608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18540cd920 con 0x7f18540d0010 2023-03-24T22:24:50.172 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.167+0000 7f185adf8700 1 -- 172.21.15.18:0/897342608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18540d2a80 con 0x7f18540d0010 2023-03-24T22:24:50.173 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.167+0000 7f18517fa700 1 -- 172.21.15.18:0/897342608 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 87553+0+0 (secure 0 0 0) 0x7f184c007450 con 0x7f18540d0010 2023-03-24T22:24:50.173 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.168+0000 7f18517fa700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1838069d80 0x7f183806c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:50.173 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.168+0000 7f18517fa700 1 -- 172.21.15.18:0/897342608 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f184c087360 con 0x7f18540d0010 2023-03-24T22:24:50.173 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.168+0000 7f1858b94700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1838069d80 0x7f183806c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:50.173 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.168+0000 7f1858b94700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1838069d80 0x7f183806c230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f18541553a0 tx=0x7f1848009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:50.174 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.172+0000 7f18517fa700 1 -- 172.21.15.18:0/897342608 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f184c0533b0 con 0x7f18540d0010 2023-03-24T22:24:50.375 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:50.375+0000 7f185adf8700 1 -- 172.21.15.18:0/897342608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f185415d220 con 0x7f18540d0010 2023-03-24T22:24:51.314 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.314+0000 7f18517fa700 1 -- 172.21.15.18:0/897342608 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7f185415d220 con 0x7f18540d0010 2023-03-24T22:24:51.318 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 -- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1838069d80 msgr2=0x7f183806c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:51.318 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1838069d80 0x7f183806c230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f18541553a0 tx=0x7f1848009040 comp rx=0 tx=0).stop 2023-03-24T22:24:51.318 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 -- 172.21.15.18:0/897342608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540d0010 msgr2=0x7f18540c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:51.318 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540d0010 0x7f18540c8c30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f184c00cc70 tx=0x7f184c00a340 comp rx=0 tx=0).stop 2023-03-24T22:24:51.318 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 -- 172.21.15.18:0/897342608 shutdown_connections 2023-03-24T22:24:51.319 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1838069d80 0x7f183806c230 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:51.319 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18540d0010 0x7f18540c8c30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:51.319 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18540b23c0 0x7f185415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:51.319 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 --2- 172.21.15.18:0/897342608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18540b5f80 0x7f18541518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:51.319 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 -- 172.21.15.18:0/897342608 >> 172.21.15.18:0/897342608 conn(0x7f1854066140 msgr2=0x7f185406b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:51.319 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 -- 172.21.15.18:0/897342608 shutdown_connections 2023-03-24T22:24:51.319 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.317+0000 7f185adf8700 1 -- 172.21.15.18:0/897342608 wait complete. 2023-03-24T22:24:51.320 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'rbd' created 2023-03-24T22:24:51.332 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-03-24T22:24:51.463 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.462+0000 7fe33c74e700 1 Processor -- start 2023-03-24T22:24:51.463 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.462+0000 7fe33c74e700 1 -- start start 2023-03-24T22:24:51.464 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe33c74e700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe3340b5f80 0x7fe3340cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:51.465 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe33c74e700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe3340d0010 0x7fe3340d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:51.465 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe33c74e700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3340d2830 con 0x7fe3340b23c0 2023-03-24T22:24:51.465 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe33c74e700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3340b2790 con 0x7fe3340d0010 2023-03-24T22:24:51.465 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe33c74e700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3340d2df0 con 0x7fe3340b5f80 2023-03-24T22:24:51.465 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe33aceb700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fe3340d0010 0x7fe3340d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:51.465 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe33a4ea700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fe3340b23c0 0x7fe3340d1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38178/0 (socket says 172.21.15.18:38178) 2023-03-24T22:24:51.466 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe33a4ea700 1 -- 172.21.15.18:0/3628703275 learned_addr learned my addr 172.21.15.18:0/3628703275 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:51.466 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe339ce9700 1 --2- 172.21.15.18:0/3628703275 >> v2:172.21.15.100:3300/0 conn(0x7fe3340b5f80 0x7fe3340cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:51.466 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe33aceb700 1 -- 172.21.15.18:0/3628703275 >> v2:172.21.15.100:3300/0 conn(0x7fe3340b5f80 msgr2=0x7fe3340cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:51.466 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.463+0000 7fe33aceb700 1 --2- 172.21.15.18:0/3628703275 >> v2:172.21.15.100:3300/0 conn(0x7fe3340b5f80 0x7fe3340cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:51.466 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe33aceb700 1 -- 172.21.15.18:0/3628703275 >> v1:172.21.15.18:6789/0 conn(0x7fe3340b23c0 legacy=0x7fe3340d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:51.466 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe3394e8700 1 -- 172.21.15.18:0/3628703275 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912374144 0 0) 0x7fe3340d2830 con 0x7fe3340b23c0 2023-03-24T22:24:51.466 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe33aceb700 1 -- 172.21.15.18:0/3628703275 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3340b6400 con 0x7fe3340d0010 2023-03-24T22:24:51.467 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe33aceb700 1 --2- 172.21.15.18:0/3628703275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340d0010 0x7fe3340d3280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe3340b5ca0 tx=0x7fe32c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86a48ec120d98c1 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:51.467 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe3394e8700 1 -- 172.21.15.18:0/3628703275 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c00e070 con 0x7fe3340d0010 2023-03-24T22:24:51.467 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe3394e8700 1 -- 172.21.15.18:0/3628703275 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c00bb20 con 0x7fe3340d0010 2023-03-24T22:24:51.467 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe3394e8700 1 -- 172.21.15.18:0/3628703275 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c012430 con 0x7fe3340d0010 2023-03-24T22:24:51.469 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe33c74e700 1 -- 172.21.15.18:0/3628703275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340d0010 msgr2=0x7fe3340d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:51.469 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe33c74e700 1 --2- 172.21.15.18:0/3628703275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340d0010 0x7fe3340d3280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe3340b5ca0 tx=0x7fe32c00b330 comp rx=0 tx=0).stop 2023-03-24T22:24:51.470 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe33c74e700 1 -- 172.21.15.18:0/3628703275 shutdown_connections 2023-03-24T22:24:51.470 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe33c74e700 1 --2- 172.21.15.18:0/3628703275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340d0010 0x7fe3340d3280 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:51.470 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe33c74e700 1 --2- 172.21.15.18:0/3628703275 >> v2:172.21.15.100:3300/0 conn(0x7fe3340b5f80 0x7fe3340cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:51.470 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.464+0000 7fe33c74e700 1 -- 172.21.15.18:0/3628703275 >> 172.21.15.18:0/3628703275 conn(0x7fe334066140 msgr2=0x7fe334066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:51.470 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.465+0000 7fe33c74e700 1 -- 172.21.15.18:0/3628703275 shutdown_connections 2023-03-24T22:24:51.470 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.465+0000 7fe33c74e700 1 -- 172.21.15.18:0/3628703275 wait complete. 2023-03-24T22:24:51.470 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.466+0000 7fe33c74e700 1 Processor -- start 2023-03-24T22:24:51.471 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.466+0000 7fe33c74e700 1 -- start start 2023-03-24T22:24:51.471 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.466+0000 7fe33c74e700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3340b23c0 0x7fe33415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:51.471 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.466+0000 7fe33c74e700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340b5f80 0x7fe334151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:51.471 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.466+0000 7fe33c74e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3340d0010 0x7fe3340c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:51.471 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.466+0000 7fe33c74e700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33415e2c0 con 0x7fe3340b5f80 2023-03-24T22:24:51.471 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.466+0000 7fe33c74e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe334151b60 con 0x7fe3340d0010 2023-03-24T22:24:51.471 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.466+0000 7fe33c74e700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341516a0 con 0x7fe3340b23c0 2023-03-24T22:24:51.471 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe339ce9700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340b5f80 0x7fe334151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:51.472 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe339ce9700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340b5f80 0x7fe334151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43858/0 (socket says 172.21.15.18:43858) 2023-03-24T22:24:51.472 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe339ce9700 1 -- 172.21.15.18:0/358333656 learned_addr learned my addr 172.21.15.18:0/358333656 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:51.472 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe33a4ea700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3340b23c0 0x7fe33415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:51.472 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe33aceb700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3340d0010 0x7fe3340c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:51.472 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe339ce9700 1 -- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3340b23c0 msgr2=0x7fe33415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:51.472 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe339ce9700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3340b23c0 0x7fe33415dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:51.472 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe339ce9700 1 -- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3340d0010 msgr2=0x7fe3340c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:51.472 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe339ce9700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3340d0010 0x7fe3340c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:51.473 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe339ce9700 1 -- 172.21.15.18:0/358333656 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c009a70 con 0x7fe3340b5f80 2023-03-24T22:24:51.473 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe339ce9700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340b5f80 0x7fe334151160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe33000e9b0 tx=0x7fe33000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:51.473 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe32affd700 1 -- 172.21.15.18:0/358333656 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330009710 con 0x7fe3340b5f80 2023-03-24T22:24:51.473 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe32affd700 1 -- 172.21.15.18:0/358333656 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33000ee50 con 0x7fe3340b5f80 2023-03-24T22:24:51.473 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe33c74e700 1 -- 172.21.15.18:0/358333656 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3340cdc00 con 0x7fe3340b5f80 2023-03-24T22:24:51.473 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe32affd700 1 -- 172.21.15.18:0/358333656 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33000ca50 con 0x7fe3340b5f80 2023-03-24T22:24:51.473 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.467+0000 7fe33c74e700 1 -- 172.21.15.18:0/358333656 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3340cd920 con 0x7fe3340b5f80 2023-03-24T22:24:51.474 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.468+0000 7fe33c74e700 1 -- 172.21.15.18:0/358333656 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3340d2a80 con 0x7fe3340b5f80 2023-03-24T22:24:51.474 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.469+0000 7fe32affd700 1 -- 172.21.15.18:0/358333656 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 87553+0+0 (secure 0 0 0) 0x7fe33000cbb0 con 0x7fe3340b5f80 2023-03-24T22:24:51.474 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.469+0000 7fe32affd700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fe318069d50 0x7fe31806c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:51.475 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.469+0000 7fe32affd700 1 -- 172.21.15.18:0/358333656 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5188+0+0 (secure 0 0 0) 0x7fe330087a30 con 0x7fe3340b5f80 2023-03-24T22:24:51.475 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.469+0000 7fe33a4ea700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fe318069d50 0x7fe31806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:51.475 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.470+0000 7fe33a4ea700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fe318069d50 0x7fe31806c200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe324007fd0 tx=0x7fe324007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:51.475 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.474+0000 7fe32affd700 1 -- 172.21.15.18:0/358333656 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fe3300537b0 con 0x7fe3340b5f80 2023-03-24T22:24:51.671 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:51.670+0000 7fe33c74e700 1 -- 172.21.15.18:0/358333656 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe33404e850 con 0x7fe3340b5f80 2023-03-24T22:24:52.316 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.315+0000 7fe32affd700 1 -- 172.21.15.18:0/358333656 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7fe3300170c0 con 0x7fe3340b5f80 2023-03-24T22:24:52.319 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.318+0000 7fe33c74e700 1 -- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fe318069d50 msgr2=0x7fe31806c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.318+0000 7fe33c74e700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fe318069d50 0x7fe31806c200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe324007fd0 tx=0x7fe324007de0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.318+0000 7fe33c74e700 1 -- 172.21.15.18:0/358333656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340b5f80 msgr2=0x7fe334151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.318+0000 7fe33c74e700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340b5f80 0x7fe334151160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe33000e9b0 tx=0x7fe33000ecc0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.318+0000 7fe33c74e700 1 -- 172.21.15.18:0/358333656 shutdown_connections 2023-03-24T22:24:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.319+0000 7fe33c74e700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fe318069d50 0x7fe31806c200 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.319+0000 7fe33c74e700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3340d0010 0x7fe3340c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.319+0000 7fe33c74e700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3340b23c0 0x7fe33415dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.321 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.319+0000 7fe33c74e700 1 --2- 172.21.15.18:0/358333656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3340b5f80 0x7fe334151160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.321 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.319+0000 7fe33c74e700 1 -- 172.21.15.18:0/358333656 >> 172.21.15.18:0/358333656 conn(0x7fe334066140 msgr2=0x7fe33415b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:52.321 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.319+0000 7fe33c74e700 1 -- 172.21.15.18:0/358333656 shutdown_connections 2023-03-24T22:24:52.321 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.319+0000 7fe33c74e700 1 -- 172.21.15.18:0/358333656 wait complete. 2023-03-24T22:24:52.321 INFO:teuthology.orchestra.run.smithi018.stderr:enabled application 'rbd' on pool 'rbd' 2023-03-24T22:24:52.336 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-03-24T22:24:52.336 INFO:tasks.ceph.mds.a:Restarting daemon 2023-03-24T22:24:52.337 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-03-24T22:24:52.339 INFO:tasks.ceph.mds.a:Started 2023-03-24T22:24:52.339 INFO:tasks.ceph.mds.c:Restarting daemon 2023-03-24T22:24:52.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-03-24T22:24:52.341 INFO:tasks.ceph.mds.c:Started 2023-03-24T22:24:52.342 INFO:tasks.ceph.mds.b:Restarting daemon 2023-03-24T22:24:52.342 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-03-24T22:24:52.344 INFO:tasks.ceph.mds.b:Started 2023-03-24T22:24:52.344 INFO:tasks.ceph.mds.d:Restarting daemon 2023-03-24T22:24:52.344 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-03-24T22:24:52.347 INFO:tasks.ceph.mds.d:Started 2023-03-24T22:24:52.347 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-03-24T22:24:52.348 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-24T22:24:52.433 INFO:tasks.ceph.mds.c.smithi018.stdout:starting mds.c at 2023-03-24T22:24:52.434 INFO:tasks.ceph.mds.a.smithi018.stdout:starting mds.a at 2023-03-24T22:24:52.438 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2023-03-24T22:24:52.441 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2023-03-24T22:24:52.495 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.494+0000 7f3a53d46700 1 Processor -- start 2023-03-24T22:24:52.496 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.494+0000 7f3a53d46700 1 -- start start 2023-03-24T22:24:52.497 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.495+0000 7f3a53d46700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c0715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.497 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.495+0000 7f3a53d46700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c15cf60 con 0x7f3a4c160530 2023-03-24T22:24:52.497 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.495+0000 7f3a53d46700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c1609a0 con 0x7f3a4c15a6e0 2023-03-24T22:24:52.497 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.495+0000 7f3a53d46700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c071af0 con 0x7f3a4c071130 2023-03-24T22:24:52.498 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.495+0000 7f3a512e1700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f3a4c160530 0x7f3a4c06e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38200/0 (socket says 172.21.15.18:38200) 2023-03-24T22:24:52.498 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.495+0000 7f3a512e1700 1 -- 172.21.15.18:0/346981322 learned_addr learned my addr 172.21.15.18:0/346981322 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:52.498 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a522e3700 1 --2- 172.21.15.18:0/346981322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c0715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.498 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a50ae0700 1 -- 172.21.15.18:0/346981322 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933272237 0 0) 0x7f3a4c15cf60 con 0x7f3a4c160530 2023-03-24T22:24:52.498 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a50ae0700 1 -- 172.21.15.18:0/346981322 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a380035b0 con 0x7f3a4c160530 2023-03-24T22:24:52.498 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a50ae0700 1 -- 172.21.15.18:0/346981322 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412006104 0 0) 0x7f3a4c1609a0 con 0x7f3a4c15a6e0 2023-03-24T22:24:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a50ae0700 1 -- 172.21.15.18:0/346981322 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a4c15cf60 con 0x7f3a4c15a6e0 2023-03-24T22:24:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a522e3700 1 -- 172.21.15.18:0/346981322 >> v1:172.21.15.100:6789/0 conn(0x7f3a4c15a6e0 legacy=0x7f3a4c0ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a522e3700 1 -- 172.21.15.18:0/346981322 >> v1:172.21.15.18:6789/0 conn(0x7f3a4c160530 legacy=0x7f3a4c06e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a522e3700 1 -- 172.21.15.18:0/346981322 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c160c80 con 0x7f3a4c071130 2023-03-24T22:24:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a522e3700 1 --2- 172.21.15.18:0/346981322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c0715b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009dc0 tx=0x7f3a4400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373dd7531d1072f4 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a50ae0700 1 -- 172.21.15.18:0/346981322 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400e070 con 0x7f3a4c071130 2023-03-24T22:24:52.499 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a50ae0700 1 -- 172.21.15.18:0/346981322 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4400bb20 con 0x7f3a4c071130 2023-03-24T22:24:52.500 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.496+0000 7f3a50ae0700 1 -- 172.21.15.18:0/346981322 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400be90 con 0x7f3a4c071130 2023-03-24T22:24:52.502 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.497+0000 7f3a53d46700 1 -- 172.21.15.18:0/346981322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 msgr2=0x7f3a4c0715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.502 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.497+0000 7f3a53d46700 1 --2- 172.21.15.18:0/346981322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c0715b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009dc0 tx=0x7f3a4400b330 comp rx=0 tx=0).stop 2023-03-24T22:24:52.502 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.497+0000 7f3a53d46700 1 -- 172.21.15.18:0/346981322 shutdown_connections 2023-03-24T22:24:52.502 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.497+0000 7f3a53d46700 1 --2- 172.21.15.18:0/346981322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c0715b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.502 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.497+0000 7f3a53d46700 1 -- 172.21.15.18:0/346981322 >> 172.21.15.18:0/346981322 conn(0x7f3a4c0cd590 msgr2=0x7f3a4c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.497+0000 7f3a53d46700 1 -- 172.21.15.18:0/346981322 shutdown_connections 2023-03-24T22:24:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.497+0000 7f3a53d46700 1 -- 172.21.15.18:0/346981322 wait complete. 2023-03-24T22:24:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.498+0000 7f3a53d46700 1 Processor -- start 2023-03-24T22:24:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.498+0000 7f3a53d46700 1 -- start start 2023-03-24T22:24:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a53d46700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c14f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a53d46700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a4c15a6e0 0x7f3a4c14fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.503 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a53d46700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c160530 0x7f3a4c1f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a53d46700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c150070 con 0x7f3a4c15a6e0 2023-03-24T22:24:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a53d46700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c1e80c0 con 0x7f3a4c160530 2023-03-24T22:24:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a53d46700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0c7440 con 0x7f3a4c071130 2023-03-24T22:24:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a512e1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a4c15a6e0 0x7f3a4c14fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a51ae2700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c14f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a51ae2700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c14f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.18:34018/0 (socket says 172.21.15.18:34018) 2023-03-24T22:24:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a512e1700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a4c15a6e0 0x7f3a4c14fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43908/0 (socket says 172.21.15.18:43908) 2023-03-24T22:24:52.504 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a51ae2700 1 -- 172.21.15.18:0/1344736381 learned_addr learned my addr 172.21.15.18:0/1344736381 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a522e3700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c160530 0x7f3a4c1f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a51ae2700 1 -- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c160530 msgr2=0x7f3a4c1f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a51ae2700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c160530 0x7f3a4c1f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a51ae2700 1 -- 172.21.15.18:0/1344736381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a4c15a6e0 msgr2=0x7f3a4c14fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a51ae2700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a4c15a6e0 0x7f3a4c14fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a51ae2700 1 -- 172.21.15.18:0/1344736381 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a44009a70 con 0x7f3a4c071130 2023-03-24T22:24:52.505 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a512e1700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a4c15a6e0 0x7f3a4c14fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.499+0000 7f3a51ae2700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c14f5f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c520 tx=0x7f3a4800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.500+0000 7f3a427fc700 1 -- 172.21.15.18:0/1344736381 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a480052d0 con 0x7f3a4c071130 2023-03-24T22:24:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.500+0000 7f3a427fc700 1 -- 172.21.15.18:0/1344736381 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a48005430 con 0x7f3a4c071130 2023-03-24T22:24:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.500+0000 7f3a53d46700 1 -- 172.21.15.18:0/1344736381 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c161240 con 0x7f3a4c071130 2023-03-24T22:24:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.500+0000 7f3a427fc700 1 -- 172.21.15.18:0/1344736381 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a480038c0 con 0x7f3a4c071130 2023-03-24T22:24:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.500+0000 7f3a53d46700 1 -- 172.21.15.18:0/1344736381 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c160f60 con 0x7f3a4c071130 2023-03-24T22:24:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.501+0000 7f3a53d46700 1 -- 172.21.15.18:0/1344736381 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4c06f090 con 0x7f3a4c071130 2023-03-24T22:24:52.506 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.501+0000 7f3a427fc700 1 -- 172.21.15.18:0/1344736381 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 4) v1 ==== 87553+0+0 (secure 0 0 0) 0x7f3a48003a20 con 0x7f3a4c071130 2023-03-24T22:24:52.507 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.501+0000 7f3a427fc700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3a30069d50 0x7f3a3006c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.507 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.501+0000 7f3a427fc700 1 -- 172.21.15.18:0/1344736381 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3a48087080 con 0x7f3a4c071130 2023-03-24T22:24:52.507 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.501+0000 7f3a512e1700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3a30069d50 0x7f3a3006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.508 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.502+0000 7f3a512e1700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3a30069d50 0x7f3a3006c200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c156670 tx=0x7f3a3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:52.508 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.506+0000 7f3a427fc700 1 -- 172.21.15.18:0/1344736381 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3a48052cc0 con 0x7f3a4c071130 2023-03-24T22:24:52.715 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.714+0000 7f3a53d46700 1 -- 172.21.15.18:0/1344736381 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a4c04e850 con 0x7f3a4c071130 2023-03-24T22:24:52.716 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.715+0000 7f3a427fc700 1 -- 172.21.15.18:0/1344736381 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14233 (secure 0 0 0) 0x7f3a480562e0 con 0x7f3a4c071130 2023-03-24T22:24:52.717 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:52.717 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":16,"fsid":"67a1ac4c-1568-462e-b9b7-d22a9a29a86a","created":"2023-03-24T22:24:39.069846+0000","modified":"2023-03-24T22:24:52.312829+0000","last_up_change":"2023-03-24T22:24:49.299255+0000","last_in_change":"2023-03-24T22:24:41.559093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-03-24T22:24:50.404940+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-03-24T22:24:50.798960+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d184f8d0-aaf7-421c-9c33-c9f32a0e9236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":114916},{"type":"v1","addr":"172.21.15.18:6821","nonce":114916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":114916},{"type":"v1","addr":"172.21.15.18:6825","nonce":114916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":114916},{"type":"v1","addr":"172.21.15.18:6831","nonce":114916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6827","nonce":114916},{"type":"v1","addr":"172.21.15.18:6829","nonce":114916}]},"public_addr":"172.21.15.18:6821/114916","cluster_addr":"172.21.15.18:6825/114916","heartbeat_back_addr":"172.21.15.18:6831/114916","heartbeat_front_addr":"172.21.15.18:6829/114916","state":["exists","up"]},{"osd":1,"uuid":"d46c78b3-52ad-41f3-8c64-3d37e9adee63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":114913},{"type":"v1","addr":"172.21.15.18:6801","nonce":114913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":114913},{"type":"v1","addr":"172.21.15.18:6803","nonce":114913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":114913},{"type":"v1","addr":"172.21.15.18:6807","nonce":114913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":114913},{"type":"v1","addr":"172.21.15.18:6805","nonce":114913}]},"public_addr":"172.21.15.18:6801/114913","cluster_addr":"172.21.15.18:6803/114913","heartbeat_back_addr":"172.21.15.18:6807/114913","heartbeat_front_addr":"172.21.15.18:6805/114913","state":["exists","up"]},{"osd":2,"uuid":"4e4e783f-f720-42f9-b394-002a79f5eb0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":114914},{"type":"v1","addr":"172.21.15.18:6809","nonce":114914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":114914},{"type":"v1","addr":"172.21.15.18:6811","nonce":114914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":114914},{"type":"v1","addr":"172.21.15.18:6815","nonce":114914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":114914},{"type":"v1","addr":"172.21.15.18:6813","nonce":114914}]},"public_addr":"172.21.15.18:6809/114914","cluster_addr":"172.21.15.18:6811/114914","heartbeat_back_addr":"172.21.15.18:6815/114914","heartbeat_front_addr":"172.21.15.18:6813/114914","state":["exists","up"]},{"osd":3,"uuid":"2a8e5486-0ae6-43c3-870e-07ad88a43340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":114915},{"type":"v1","addr":"172.21.15.18:6817","nonce":114915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":114915},{"type":"v1","addr":"172.21.15.18:6820","nonce":114915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":114915},{"type":"v1","addr":"172.21.15.18:6828","nonce":114915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":114915},{"type":"v1","addr":"172.21.15.18:6824","nonce":114915}]},"public_addr":"172.21.15.18:6817/114915","cluster_addr":"172.21.15.18:6820/114915","heartbeat_back_addr":"172.21.15.18:6828/114915","heartbeat_front_addr":"172.21.15.18:6824/114915","state":["exists","up"]},{"osd":4,"uuid":"9a82df25-c818-47d7-8a9e-3e6fd55cd4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":115214},{"type":"v1","addr":"172.21.15.100:6801","nonce":115214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":115214},{"type":"v1","addr":"172.21.15.100:6803","nonce":115214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":115214},{"type":"v1","addr":"172.21.15.100:6807","nonce":115214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":115214},{"type":"v1","addr":"172.21.15.100:6805","nonce":115214}]},"public_addr":"172.21.15.100:6801/115214","cluster_addr":"172.21.15.100:6803/115214","heartbeat_back_addr":"172.21.15.100:6807/115214","heartbeat_front_addr":"172.21.15.100:6805/115214","state":["exists","up"]},{"osd":5,"uuid":"ae37e2cd-d079-4962-9bd7-628aa5a7e14d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":115215},{"type":"v1","addr":"172.21.15.100:6809","nonce":115215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":115215},{"type":"v1","addr":"172.21.15.100:6811","nonce":115215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":115215},{"type":"v1","addr":"172.21.15.100:6815","nonce":115215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":115215},{"type":"v1","addr":"172.21.15.100:6813","nonce":115215}]},"public_addr":"172.21.15.100:6809/115215","cluster_addr":"172.21.15.100:6811/115215","heartbeat_back_addr":"172.21.15.100:6815/115215","heartbeat_front_addr":"172.21.15.100:6813/115215","state":["exists","up"]},{"osd":6,"uuid":"e3574199-2966-431b-ab0b-25efed52c749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":115255},{"type":"v1","addr":"172.21.15.100:6825","nonce":115255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":115255},{"type":"v1","addr":"172.21.15.100:6827","nonce":115255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":115255},{"type":"v1","addr":"172.21.15.100:6831","nonce":115255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":115255},{"type":"v1","addr":"172.21.15.100:6829","nonce":115255}]},"public_addr":"172.21.15.100:6825/115255","cluster_addr":"172.21.15.100:6827/115255","heartbeat_back_addr":"172.21.15.100:6831/115255","heartbeat_front_addr":"172.21.15.100:6829/115255","state":["exists","up"]},{"osd":7,"uuid":"5a2a90ec-837f-4560-8a8e-b3e82eab590b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":115216},{"type":"v1","addr":"172.21.15.100:6817","nonce":115216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":115216},{"type":"v1","addr":"172.21.15.100:6819","nonce":115216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":115216},{"type":"v1","addr":"172.21.15.100:6823","nonce":115216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":115216},{"type":"v1","addr":"172.21.15.100:6821","nonce":115216}]},"public_addr":"172.21.15.100:6817/115216","cluster_addr":"172.21.15.100:6819/115216","heartbeat_back_addr":"172.21.15.100:6823/115216","heartbeat_front_addr":"172.21.15.100:6821/115216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.718826+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.668044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.678787+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.750169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.685071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.741353+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.757517+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.737387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T22:24:52.718 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 -- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3a30069d50 msgr2=0x7f3a3006c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.719 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3a30069d50 0x7f3a3006c200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c156670 tx=0x7f3a3c009040 comp rx=0 tx=0).stop 2023-03-24T22:24:52.719 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 -- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 msgr2=0x7f3a4c14f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.719 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c14f5f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c520 tx=0x7f3a4800c830 comp rx=0 tx=0).stop 2023-03-24T22:24:52.719 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 -- 172.21.15.18:0/1344736381 shutdown_connections 2023-03-24T22:24:52.720 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3a30069d50 0x7f3a3006c200 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.720 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c160530 0x7f3a4c1f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.720 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c071130 0x7f3a4c14f5f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.720 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 --2- 172.21.15.18:0/1344736381 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3a4c15a6e0 0x7f3a4c14fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.720 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 -- 172.21.15.18:0/1344736381 >> 172.21.15.18:0/1344736381 conn(0x7f3a4c0cd590 msgr2=0x7f3a4c15b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:52.721 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 -- 172.21.15.18:0/1344736381 shutdown_connections 2023-03-24T22:24:52.721 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.717+0000 7f3a53d46700 1 -- 172.21.15.18:0/1344736381 wait complete. 2023-03-24T22:24:52.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-03-24T22:24:50.404940+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-03-24T22:24:50.798960+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-24T22:24:52.732 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-24T22:24:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.862+0000 7fc9c0d6a700 1 Processor -- start 2023-03-24T22:24:52.864 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.862+0000 7fc9c0d6a700 1 -- start start 2023-03-24T22:24:52.866 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.863+0000 7fc9c0d6a700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 0x7fc9bc06ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.866 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.863+0000 7fc9c0d6a700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc9bc0cc970 0x7fc9bc0ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.867 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.863+0000 7fc9c0d6a700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9bc15bb60 con 0x7fc9bc0cff20 2023-03-24T22:24:52.867 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.863+0000 7fc9c0d6a700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9bc15ac70 con 0x7fc9bc0cc970 2023-03-24T22:24:52.867 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.863+0000 7fc9c0d6a700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc15af50 con 0x7fc9bc06faf0 2023-03-24T22:24:52.867 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.863+0000 7fc9bad9d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc9bc0cc970 0x7fc9bc0ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.867 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.863+0000 7fc9ba59c700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fc9bc0cff20 0x7fc9bc0d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38216/0 (socket says 172.21.15.18:38216) 2023-03-24T22:24:52.868 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.863+0000 7fc9bad9d700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fc9bc0cc970 0x7fc9bc0ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43922/0 (socket says 172.21.15.18:43922) 2023-03-24T22:24:52.868 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.863+0000 7fc9ba59c700 1 -- 172.21.15.18:0/1407917502 learned_addr learned my addr 172.21.15.18:0/1407917502 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:52.868 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.863+0000 7fc9b9d9b700 1 --2- 172.21.15.18:0/1407917502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 0x7fc9bc06ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.868 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.864+0000 7fc9bad9d700 1 -- 172.21.15.18:0/1407917502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 msgr2=0x7fc9bc06ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.864+0000 7fc9bad9d700 1 --2- 172.21.15.18:0/1407917502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 0x7fc9bc06ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.864+0000 7fc9bad9d700 1 -- 172.21.15.18:0/1407917502 >> v1:172.21.15.18:6789/0 conn(0x7fc9bc0cff20 legacy=0x7fc9bc0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.864+0000 7fc9bad9d700 1 -- 172.21.15.18:0/1407917502 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc15b230 con 0x7fc9bc0cc970 2023-03-24T22:24:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.864+0000 7fc9b9d9b700 1 --2- 172.21.15.18:0/1407917502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 0x7fc9bc06ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.864+0000 7fc9b959a700 1 -- 172.21.15.18:0/1407917502 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2440503870 0 0) 0x7fc9bc15bb60 con 0x7fc9bc0cff20 2023-03-24T22:24:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.864+0000 7fc9bad9d700 1 --2- 172.21.15.18:0/1407917502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 0x7fc9bc0ccdf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009d90 tx=0x7fc9b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dae1c9e4cfa1379 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:52.869 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.864+0000 7fc9b959a700 1 -- 172.21.15.18:0/1407917502 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400e070 con 0x7fc9bc0cc970 2023-03-24T22:24:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.864+0000 7fc9b959a700 1 -- 172.21.15.18:0/1407917502 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b400bb20 con 0x7fc9bc0cc970 2023-03-24T22:24:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.864+0000 7fc9b959a700 1 -- 172.21.15.18:0/1407917502 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4012430 con 0x7fc9bc0cc970 2023-03-24T22:24:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.865+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/1407917502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 msgr2=0x7fc9bc0ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.870 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.865+0000 7fc9c0d6a700 1 --2- 172.21.15.18:0/1407917502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 0x7fc9bc0ccdf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009d90 tx=0x7fc9b400b330 comp rx=0 tx=0).stop 2023-03-24T22:24:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.865+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/1407917502 shutdown_connections 2023-03-24T22:24:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.865+0000 7fc9c0d6a700 1 --2- 172.21.15.18:0/1407917502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 0x7fc9bc0ccdf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.865+0000 7fc9c0d6a700 1 --2- 172.21.15.18:0/1407917502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 0x7fc9bc06ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.865+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/1407917502 >> 172.21.15.18:0/1407917502 conn(0x7fc9bc06e360 msgr2=0x7fc9bc06e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.865+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/1407917502 shutdown_connections 2023-03-24T22:24:52.872 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.865+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/1407917502 wait complete. 2023-03-24T22:24:52.874 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.866+0000 7fc9c0d6a700 1 Processor -- start 2023-03-24T22:24:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.866+0000 7fc9c0d6a700 1 -- start start 2023-03-24T22:24:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9c0d6a700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 0x7fc9bc0b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9c0d6a700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 0x7fc9bc0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.875 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9c0d6a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9bc0cff20 0x7fc9bc1f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.876 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9c0d6a700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0c91f0 con 0x7fc9bc0cc970 2023-03-24T22:24:52.876 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9c0d6a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc156910 con 0x7fc9bc0cff20 2023-03-24T22:24:52.876 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9c0d6a700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0b6860 con 0x7fc9bc06faf0 2023-03-24T22:24:52.876 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9b9d9b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 0x7fc9bc0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.876 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9b9d9b700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 0x7fc9bc0ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43936/0 (socket says 172.21.15.18:43936) 2023-03-24T22:24:52.876 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9b9d9b700 1 -- 172.21.15.18:0/2553582270 learned_addr learned my addr 172.21.15.18:0/2553582270 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:52.876 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9ba59c700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 0x7fc9bc0b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.877 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9bad9d700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9bc0cff20 0x7fc9bc1f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.877 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9b9d9b700 1 -- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 msgr2=0x7fc9bc0b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.877 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9b9d9b700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 0x7fc9bc0b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.877 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9b9d9b700 1 -- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9bc0cff20 msgr2=0x7fc9bc1f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:52.877 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9b9d9b700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9bc0cff20 0x7fc9bc1f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:52.877 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9b9d9b700 1 -- 172.21.15.18:0/2553582270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac0094a0 con 0x7fc9bc0cc970 2023-03-24T22:24:52.877 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.867+0000 7fc9b9d9b700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 0x7fc9bc0ca890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00ebe0 tx=0x7fc9ac00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:52.878 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.868+0000 7fc9aaffd700 1 -- 172.21.15.18:0/2553582270 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac009760 con 0x7fc9bc0cc970 2023-03-24T22:24:52.878 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.868+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/2553582270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b4009a70 con 0x7fc9bc0cc970 2023-03-24T22:24:52.878 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.868+0000 7fc9aaffd700 1 -- 172.21.15.18:0/2553582270 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ac00c850 con 0x7fc9bc0cc970 2023-03-24T22:24:52.878 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.868+0000 7fc9aaffd700 1 -- 172.21.15.18:0/2553582270 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00cbe0 con 0x7fc9bc0cc970 2023-03-24T22:24:52.878 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.868+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/2553582270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc15b7f0 con 0x7fc9bc0cc970 2023-03-24T22:24:52.878 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.868+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/2553582270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc15b510 con 0x7fc9bc0cc970 2023-03-24T22:24:52.878 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.869+0000 7fc9aaffd700 1 -- 172.21.15.18:0/2553582270 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 87553+0+0 (secure 0 0 0) 0x7fc9ac00cd40 con 0x7fc9bc0cc970 2023-03-24T22:24:52.879 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.869+0000 7fc9aaffd700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fc99c069d50 0x7fc99c06c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:52.879 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.870+0000 7fc9aaffd700 1 -- 172.21.15.18:0/2553582270 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc9ac087b50 con 0x7fc9bc0cc970 2023-03-24T22:24:52.879 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.874+0000 7fc9aaffd700 1 -- 172.21.15.18:0/2553582270 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fc9ac053840 con 0x7fc9bc0cc970 2023-03-24T22:24:52.887 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.886+0000 7fc9ba59c700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fc99c069d50 0x7fc99c06c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:52.887 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:52.886+0000 7fc9ba59c700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fc99c069d50 0x7fc99c06c200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0007fd0 tx=0x7fc9b0007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:53.071 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.069+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/2553582270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc9bc158460 con 0x7fc9bc0cc970 2023-03-24T22:24:53.071 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.070+0000 7fc9aaffd700 1 -- 172.21.15.18:0/2553582270 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc9ac056e60 con 0x7fc9bc0cc970 2023-03-24T22:24:53.072 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 8 2023-03-24T22:24:53.074 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fc99c069d50 msgr2=0x7fc99c06c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.074 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fc99c069d50 0x7fc99c06c200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0007fd0 tx=0x7fc9b0007de0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.074 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/2553582270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 msgr2=0x7fc9bc0ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.075 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 0x7fc9bc0ca890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00ebe0 tx=0x7fc9ac00eef0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.075 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/2553582270 shutdown_connections 2023-03-24T22:24:53.075 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fc99c069d50 0x7fc99c06c200 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.075 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9bc0cff20 0x7fc9bc1f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.076 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9bc06faf0 0x7fc9bc0b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.076 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 --2- 172.21.15.18:0/2553582270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9bc0cc970 0x7fc9bc0ca890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.076 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/2553582270 >> 172.21.15.18:0/2553582270 conn(0x7fc9bc06e360 msgr2=0x7fc9bc15d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:53.077 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/2553582270 shutdown_connections 2023-03-24T22:24:53.077 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.072+0000 7fc9c0d6a700 1 -- 172.21.15.18:0/2553582270 wait complete. 2023-03-24T22:24:53.085 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-24T22:24:53.216 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.215+0000 7fd22e817700 1 Processor -- start 2023-03-24T22:24:53.217 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.215+0000 7fd22e817700 1 -- start start 2023-03-24T22:24:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.216+0000 7fd22e817700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 0x7fd2280cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.216+0000 7fd22e817700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22815e280 con 0x7fd228159990 2023-03-24T22:24:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.216+0000 7fd22e817700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22806bb90 con 0x7fd22806b5b0 2023-03-24T22:24:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.216+0000 7fd22e817700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280cf1a0 con 0x7fd22806d910 2023-03-24T22:24:53.218 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.216+0000 7fd227fff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fd228159990 0x7fd22815db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38228/0 (socket says 172.21.15.18:38228) 2023-03-24T22:24:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.216+0000 7fd227fff700 1 -- 172.21.15.18:0/1714204822 learned_addr learned my addr 172.21.15.18:0/1714204822 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.216+0000 7fd22cdb4700 1 --2- 172.21.15.18:0/1714204822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 0x7fd2280cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.217+0000 7fd226ffd700 1 -- 172.21.15.18:0/1714204822 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977011189 0 0) 0x7fd22806bb90 con 0x7fd22806b5b0 2023-03-24T22:24:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.217+0000 7fd226ffd700 1 -- 172.21.15.18:0/1714204822 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2100035b0 con 0x7fd22806b5b0 2023-03-24T22:24:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.217+0000 7fd22cdb4700 1 -- 172.21.15.18:0/1714204822 >> v1:172.21.15.100:6789/0 conn(0x7fd22806b5b0 legacy=0x7fd22806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.217+0000 7fd22cdb4700 1 -- 172.21.15.18:0/1714204822 >> v1:172.21.15.18:6789/0 conn(0x7fd228159990 legacy=0x7fd22815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.219 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.217+0000 7fd22cdb4700 1 -- 172.21.15.18:0/1714204822 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22806be70 con 0x7fd22806d910 2023-03-24T22:24:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.217+0000 7fd22cdb4700 1 --2- 172.21.15.18:0/1714204822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 0x7fd2280cec60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd220009dc0 tx=0x7fd22000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81993dc4e0e9b8c7 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.217+0000 7fd226ffd700 1 -- 172.21.15.18:0/1714204822 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22000e070 con 0x7fd22806d910 2023-03-24T22:24:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.217+0000 7fd226ffd700 1 -- 172.21.15.18:0/1714204822 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22000bb20 con 0x7fd22806d910 2023-03-24T22:24:53.220 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.217+0000 7fd226ffd700 1 -- 172.21.15.18:0/1714204822 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22000be90 con 0x7fd22806d910 2023-03-24T22:24:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.218+0000 7fd22e817700 1 -- 172.21.15.18:0/1714204822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 msgr2=0x7fd2280cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.218+0000 7fd22e817700 1 --2- 172.21.15.18:0/1714204822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 0x7fd2280cec60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd220009dc0 tx=0x7fd22000b330 comp rx=0 tx=0).stop 2023-03-24T22:24:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.218+0000 7fd22e817700 1 -- 172.21.15.18:0/1714204822 shutdown_connections 2023-03-24T22:24:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.218+0000 7fd22e817700 1 --2- 172.21.15.18:0/1714204822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 0x7fd2280cec60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.218+0000 7fd22e817700 1 -- 172.21.15.18:0/1714204822 >> 172.21.15.18:0/1714204822 conn(0x7fd2280cd590 msgr2=0x7fd2280cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:53.222 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.218+0000 7fd22e817700 1 -- 172.21.15.18:0/1714204822 shutdown_connections 2023-03-24T22:24:53.222 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.218+0000 7fd22e817700 1 -- 172.21.15.18:0/1714204822 wait complete. 2023-03-24T22:24:53.222 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.219+0000 7fd22e817700 1 Processor -- start 2023-03-24T22:24:53.222 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.220+0000 7fd22e817700 1 -- start start 2023-03-24T22:24:53.223 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd22e817700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd22806b5b0 0x7fd228150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd22e817700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 0x7fd2281582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd22e817700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd228159990 0x7fd2280b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd22e817700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280b43c0 con 0x7fd228159990 2023-03-24T22:24:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd22e817700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd228154e10 con 0x7fd22806b5b0 2023-03-24T22:24:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd22e817700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2281550f0 con 0x7fd22806d910 2023-03-24T22:24:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd227fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd22806b5b0 0x7fd228150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.224 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd2277fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 0x7fd2281582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.225 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd22cdb4700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd228159990 0x7fd2280b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.225 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd227fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd22806b5b0 0x7fd228150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.18:43490/0 (socket says 172.21.15.18:43490) 2023-03-24T22:24:53.225 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd227fff700 1 -- 172.21.15.18:0/2795932763 learned_addr learned my addr 172.21.15.18:0/2795932763 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:53.225 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd227fff700 1 -- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 msgr2=0x7fd2281582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.225 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd227fff700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 0x7fd2281582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.225 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd227fff700 1 -- 172.21.15.18:0/2795932763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd228159990 msgr2=0x7fd2280b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.226 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd227fff700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd228159990 0x7fd2280b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.226 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd227fff700 1 -- 172.21.15.18:0/2795932763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd220009a70 con 0x7fd22806b5b0 2023-03-24T22:24:53.226 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.221+0000 7fd2277fe700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 0x7fd2281582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:53.226 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.222+0000 7fd227fff700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd22806b5b0 0x7fd228150cf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd21800c520 tx=0x7fd21800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:53.226 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.222+0000 7fd224ff9700 1 -- 172.21.15.18:0/2795932763 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218005d70 con 0x7fd22806b5b0 2023-03-24T22:24:53.226 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.222+0000 7fd224ff9700 1 -- 172.21.15.18:0/2795932763 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2180070d0 con 0x7fd22806b5b0 2023-03-24T22:24:53.226 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.222+0000 7fd22e817700 1 -- 172.21.15.18:0/2795932763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22815d370 con 0x7fd22806b5b0 2023-03-24T22:24:53.227 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.222+0000 7fd224ff9700 1 -- 172.21.15.18:0/2795932763 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218007440 con 0x7fd22806b5b0 2023-03-24T22:24:53.227 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.222+0000 7fd22e817700 1 -- 172.21.15.18:0/2795932763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22806c150 con 0x7fd22806b5b0 2023-03-24T22:24:53.228 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.223+0000 7fd224ff9700 1 -- 172.21.15.18:0/2795932763 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 87553+0+0 (secure 0 0 0) 0x7fd2180075a0 con 0x7fd22806b5b0 2023-03-24T22:24:53.228 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.223+0000 7fd22e817700 1 -- 172.21.15.18:0/2795932763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22815d650 con 0x7fd22806b5b0 2023-03-24T22:24:53.228 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.223+0000 7fd224ff9700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fd214069d50 0x7fd21406c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.228 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.224+0000 7fd224ff9700 1 -- 172.21.15.18:0/2795932763 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd218086ea0 con 0x7fd22806b5b0 2023-03-24T22:24:53.228 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.224+0000 7fd2277fe700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fd214069d50 0x7fd21406c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.228 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.224+0000 7fd2277fe700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fd214069d50 0x7fd21406c200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd2281553f0 tx=0x7fd21c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:53.229 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.228+0000 7fd224ff9700 1 -- 172.21.15.18:0/2795932763 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd21804c160 con 0x7fd22806b5b0 2023-03-24T22:24:53.426 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.425+0000 7fd22e817700 1 -- 172.21.15.18:0/2795932763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd2280b27a0 con 0x7fd22806b5b0 2023-03-24T22:24:53.427 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.426+0000 7fd224ff9700 1 -- 172.21.15.18:0/2795932763 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd218052b90 con 0x7fd22806b5b0 2023-03-24T22:24:53.427 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2023-03-24T22:24:53.429 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.428+0000 7fd22e817700 1 -- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fd214069d50 msgr2=0x7fd21406c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.428+0000 7fd22e817700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fd214069d50 0x7fd21406c200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd2281553f0 tx=0x7fd21c009040 comp rx=0 tx=0).stop 2023-03-24T22:24:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.428+0000 7fd22e817700 1 -- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd22806b5b0 msgr2=0x7fd228150cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.428+0000 7fd22e817700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd22806b5b0 0x7fd228150cf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd21800c520 tx=0x7fd21800c830 comp rx=0 tx=0).stop 2023-03-24T22:24:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.429+0000 7fd22e817700 1 -- 172.21.15.18:0/2795932763 shutdown_connections 2023-03-24T22:24:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.429+0000 7fd22e817700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fd214069d50 0x7fd21406c200 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.429+0000 7fd22e817700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd228159990 0x7fd2280b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.429+0000 7fd22e817700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd22806b5b0 0x7fd228150cf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.429+0000 7fd22e817700 1 --2- 172.21.15.18:0/2795932763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd22806d910 0x7fd2281582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.429+0000 7fd22e817700 1 -- 172.21.15.18:0/2795932763 >> 172.21.15.18:0/2795932763 conn(0x7fd2280cd590 msgr2=0x7fd2281555b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.429+0000 7fd22e817700 1 -- 172.21.15.18:0/2795932763 shutdown_connections 2023-03-24T22:24:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.429+0000 7fd22e817700 1 -- 172.21.15.18:0/2795932763 wait complete. 2023-03-24T22:24:53.442 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-24T22:24:53.571 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.567+0000 7fb4a9ec8700 1 Processor -- start 2023-03-24T22:24:53.571 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.567+0000 7fb4a9ec8700 1 -- start start 2023-03-24T22:24:53.571 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a9ec8700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40b23c0 0x7fb4a40b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.571 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a9ec8700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb4a40d1710 0x7fb4a40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.571 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a9ec8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a40d2830 con 0x7fb4a40d1b60 2023-03-24T22:24:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a9ec8700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a40b2790 con 0x7fb4a40d1710 2023-03-24T22:24:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a9ec8700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40d2df0 con 0x7fb4a40b23c0 2023-03-24T22:24:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a3fff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fb4a40d1b60 0x7fb4a40d3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38242/0 (socket says 172.21.15.18:38242) 2023-03-24T22:24:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a3fff700 1 -- 172.21.15.18:0/186090064 learned_addr learned my addr 172.21.15.18:0/186090064 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a37fe700 1 --2- 172.21.15.18:0/186090064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40b23c0 0x7fb4a40b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a2ffd700 1 --2- 172.21.15.18:0/186090064 >> v2:172.21.15.100:3300/0 conn(0x7fb4a40d1710 0x7fb4a40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a27fc700 1 -- 172.21.15.18:0/186090064 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298815902 0 0) 0x7fb4a40d2830 con 0x7fb4a40d1b60 2023-03-24T22:24:53.572 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a27fc700 1 -- 172.21.15.18:0/186090064 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4900035b0 con 0x7fb4a40d1b60 2023-03-24T22:24:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a2ffd700 1 -- 172.21.15.18:0/186090064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40b23c0 msgr2=0x7fb4a40b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a2ffd700 1 --2- 172.21.15.18:0/186090064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40b23c0 0x7fb4a40b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a2ffd700 1 -- 172.21.15.18:0/186090064 >> v1:172.21.15.18:6789/0 conn(0x7fb4a40d1b60 legacy=0x7fb4a40d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a2ffd700 1 -- 172.21.15.18:0/186090064 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a40cd920 con 0x7fb4a40d1710 2023-03-24T22:24:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.568+0000 7fb4a37fe700 1 --2- 172.21.15.18:0/186090064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40b23c0 0x7fb4a40b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a2ffd700 1 --2- 172.21.15.18:0/186090064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a40d1710 0x7fb4a40cfad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009840 tx=0x7fb48c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a7d23fcb1ae910 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:53.573 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a27fc700 1 -- 172.21.15.18:0/186090064 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c011830 con 0x7fb4a40d1710 2023-03-24T22:24:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a27fc700 1 -- 172.21.15.18:0/186090064 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c011990 con 0x7fb4a40d1710 2023-03-24T22:24:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a27fc700 1 -- 172.21.15.18:0/186090064 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c011d20 con 0x7fb4a40d1710 2023-03-24T22:24:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/186090064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a40d1710 msgr2=0x7fb4a40cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a9ec8700 1 --2- 172.21.15.18:0/186090064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a40d1710 0x7fb4a40cfad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009840 tx=0x7fb48c009b50 comp rx=0 tx=0).stop 2023-03-24T22:24:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/186090064 shutdown_connections 2023-03-24T22:24:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a9ec8700 1 --2- 172.21.15.18:0/186090064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40b23c0 0x7fb4a40b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.574 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a9ec8700 1 --2- 172.21.15.18:0/186090064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a40d1710 0x7fb4a40cfad0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/186090064 >> 172.21.15.18:0/186090064 conn(0x7fb4a4066140 msgr2=0x7fb4a4066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/186090064 shutdown_connections 2023-03-24T22:24:53.575 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.569+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/186090064 wait complete. 2023-03-24T22:24:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.571+0000 7fb4a9ec8700 1 Processor -- start 2023-03-24T22:24:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.571+0000 7fb4a9ec8700 1 -- start start 2023-03-24T22:24:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.571+0000 7fb4a9ec8700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4a40b23c0 0x7fb4a40cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.571+0000 7fb4a9ec8700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40d1710 0x7fb4a4159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.571+0000 7fb4a9ec8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a40ccea0 0x7fb4a41f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.577 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.571+0000 7fb4a9ec8700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40cc9a0 con 0x7fb4a40b23c0 2023-03-24T22:24:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.571+0000 7fb4a9ec8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a415aa50 con 0x7fb4a40ccea0 2023-03-24T22:24:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.571+0000 7fb4a9ec8700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a41e75a0 con 0x7fb4a40d1710 2023-03-24T22:24:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a37fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4a40b23c0 0x7fb4a40cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a37fe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4a40b23c0 0x7fb4a40cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43954/0 (socket says 172.21.15.18:43954) 2023-03-24T22:24:53.578 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a37fe700 1 -- 172.21.15.18:0/2073538173 learned_addr learned my addr 172.21.15.18:0/2073538173 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a3fff700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a40ccea0 0x7fb4a41f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a2ffd700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40d1710 0x7fb4a4159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a37fe700 1 -- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40d1710 msgr2=0x7fb4a4159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a37fe700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40d1710 0x7fb4a4159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a37fe700 1 -- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a40ccea0 msgr2=0x7fb4a41f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a37fe700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a40ccea0 0x7fb4a41f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a37fe700 1 -- 172.21.15.18:0/2073538173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4940094a0 con 0x7fb4a40b23c0 2023-03-24T22:24:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a37fe700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4a40b23c0 0x7fb4a40cb600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb49400ebe0 tx=0x7fb49400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb49bfff700 1 -- 172.21.15.18:0/2073538173 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494009760 con 0x7fb4a40b23c0 2023-03-24T22:24:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb49bfff700 1 -- 172.21.15.18:0/2073538173 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49400c850 con 0x7fb4a40b23c0 2023-03-24T22:24:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/2073538173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c0094a0 con 0x7fb4a40b23c0 2023-03-24T22:24:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb49bfff700 1 -- 172.21.15.18:0/2073538173 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49400cbe0 con 0x7fb4a40b23c0 2023-03-24T22:24:53.580 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.572+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/2073538173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a40d2a80 con 0x7fb4a40b23c0 2023-03-24T22:24:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.573+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/2073538173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a40cdc00 con 0x7fb4a40b23c0 2023-03-24T22:24:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.576+0000 7fb49bfff700 1 -- 172.21.15.18:0/2073538173 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 87553+0+0 (secure 0 0 0) 0x7fb49400cd40 con 0x7fb4a40b23c0 2023-03-24T22:24:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.576+0000 7fb49bfff700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fb484069d50 0x7fb48406c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.576+0000 7fb49bfff700 1 -- 172.21.15.18:0/2073538173 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb494088200 con 0x7fb4a40b23c0 2023-03-24T22:24:53.581 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.576+0000 7fb4a2ffd700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fb484069d50 0x7fb48406c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.582 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.577+0000 7fb4a2ffd700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fb484069d50 0x7fb48406c200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb48c000c00 tx=0x7fb48c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:53.582 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.580+0000 7fb49bfff700 1 -- 172.21.15.18:0/2073538173 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fb494057a80 con 0x7fb4a40b23c0 2023-03-24T22:24:53.584 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.583+0000 7fb49bfff700 1 -- 172.21.15.18:0/2073538173 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fb494054420 con 0x7fb4a40b23c0 2023-03-24T22:24:53.828 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.827+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/2073538173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4a40cf530 con 0x7fb4a40b23c0 2023-03-24T22:24:53.828 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.827+0000 7fb49bfff700 1 -- 172.21.15.18:0/2073538173 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7fb494017de0 con 0x7fb4a40b23c0 2023-03-24T22:24:53.829 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:53.829 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2267092624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2267092624},{"type":"v1","addr":"172.21.15.100:6835","nonce":2267092624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4071500859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":4071500859},{"type":"v1","addr":"172.21.15.100:6837","nonce":4071500859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6834/455617156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":455617156},{"type":"v1","addr":"172.21.15.18:6834","nonce":455617156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/351746815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6833","nonce":351746815},{"type":"v1","addr":"172.21.15.18:6835","nonce":351746815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-03-24T22:24:53.830 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fb484069d50 msgr2=0x7fb48406c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.831 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fb484069d50 0x7fb48406c200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb48c000c00 tx=0x7fb48c008030 comp rx=0 tx=0).stop 2023-03-24T22:24:53.831 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/2073538173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4a40b23c0 msgr2=0x7fb4a40cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.831 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4a40b23c0 0x7fb4a40cb600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb49400ebe0 tx=0x7fb49400eef0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.831 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/2073538173 shutdown_connections 2023-03-24T22:24:53.831 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fb484069d50 0x7fb48406c200 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.831 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4a40b23c0 0x7fb4a40cb600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.831 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a40ccea0 0x7fb4a41f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.832 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 --2- 172.21.15.18:0/2073538173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4a40d1710 0x7fb4a4159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.832 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/2073538173 >> 172.21.15.18:0/2073538173 conn(0x7fb4a4066140 msgr2=0x7fb4a415b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:53.832 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/2073538173 shutdown_connections 2023-03-24T22:24:53.832 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.829+0000 7fb4a9ec8700 1 -- 172.21.15.18:0/2073538173 wait complete. 2023-03-24T22:24:53.832 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 3 2023-03-24T22:24:53.843 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-24T22:24:53.968 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.967+0000 7f9c899dc700 1 Processor -- start 2023-03-24T22:24:53.968 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.967+0000 7f9c899dc700 1 -- start start 2023-03-24T22:24:53.970 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c899dc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 0x7f9c840b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.970 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c899dc700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9c840d1710 0x7f9c840cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.970 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c899dc700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c840d2830 con 0x7f9c840d1b60 2023-03-24T22:24:53.970 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c899dc700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c840b2790 con 0x7f9c840d1710 2023-03-24T22:24:53.971 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c899dc700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c840d2df0 con 0x7f9c840b23c0 2023-03-24T22:24:53.971 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c837fe700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f9c840d1b60 0x7f9c840d3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38252/0 (socket says 172.21.15.18:38252) 2023-03-24T22:24:53.971 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c837fe700 1 -- 172.21.15.18:0/3541842645 learned_addr learned my addr 172.21.15.18:0/3541842645 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:53.971 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c827fc700 1 --2- 172.21.15.18:0/3541842645 >> v2:172.21.15.100:3300/0 conn(0x7f9c840d1710 0x7f9c840cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.971 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c82ffd700 1 --2- 172.21.15.18:0/3541842645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 0x7f9c840b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.971 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c81ffb700 1 -- 172.21.15.18:0/3541842645 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861137470 0 0) 0x7f9c840d2830 con 0x7f9c840d1b60 2023-03-24T22:24:53.971 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c81ffb700 1 -- 172.21.15.18:0/3541842645 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c6c0035b0 con 0x7f9c840d1b60 2023-03-24T22:24:53.972 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c827fc700 1 -- 172.21.15.18:0/3541842645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 msgr2=0x7f9c840b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:53.972 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c827fc700 1 --2- 172.21.15.18:0/3541842645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 0x7f9c840b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.973 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c827fc700 1 -- 172.21.15.18:0/3541842645 >> v1:172.21.15.18:6789/0 conn(0x7f9c840d1b60 legacy=0x7f9c840d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.973 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c827fc700 1 -- 172.21.15.18:0/3541842645 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c840cd920 con 0x7f9c840d1710 2023-03-24T22:24:53.973 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.968+0000 7f9c827fc700 1 --2- 172.21.15.18:0/3541842645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c840d1710 0x7f9c840cfad0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009840 tx=0x7f9c74009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be320b9a166a3a72 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:53.973 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c81ffb700 1 -- 172.21.15.18:0/3541842645 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74011830 con 0x7f9c840d1710 2023-03-24T22:24:53.973 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c81ffb700 1 -- 172.21.15.18:0/3541842645 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c74011990 con 0x7f9c840d1710 2023-03-24T22:24:53.974 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c81ffb700 1 -- 172.21.15.18:0/3541842645 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74011d20 con 0x7f9c840d1710 2023-03-24T22:24:53.974 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c899dc700 1 -- 172.21.15.18:0/3541842645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c840d1710 msgr2=0x7f9c840cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.974 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c899dc700 1 --2- 172.21.15.18:0/3541842645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c840d1710 0x7f9c840cfad0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009840 tx=0x7f9c74009b50 comp rx=0 tx=0).stop 2023-03-24T22:24:53.974 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c899dc700 1 -- 172.21.15.18:0/3541842645 shutdown_connections 2023-03-24T22:24:53.974 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c899dc700 1 --2- 172.21.15.18:0/3541842645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 0x7f9c840b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.974 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c899dc700 1 --2- 172.21.15.18:0/3541842645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c840d1710 0x7f9c840cfad0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.974 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c899dc700 1 -- 172.21.15.18:0/3541842645 >> 172.21.15.18:0/3541842645 conn(0x7f9c84066140 msgr2=0x7f9c84066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:53.974 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c899dc700 1 -- 172.21.15.18:0/3541842645 shutdown_connections 2023-03-24T22:24:53.975 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.969+0000 7f9c899dc700 1 -- 172.21.15.18:0/3541842645 wait complete. 2023-03-24T22:24:53.977 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.970+0000 7f9c899dc700 1 Processor -- start 2023-03-24T22:24:53.977 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.971+0000 7f9c899dc700 1 -- start start 2023-03-24T22:24:53.977 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.971+0000 7f9c899dc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 0x7f9c841472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.977 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.971+0000 7f9c899dc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c840d1710 0x7f9c84148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.977 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.971+0000 7f9c899dc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c840cb2b0 0x7f9c840cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.978 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.971+0000 7f9c899dc700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c840cbc90 con 0x7f9c840d1710 2023-03-24T22:24:53.978 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.971+0000 7f9c899dc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c84148580 con 0x7f9c840cb2b0 2023-03-24T22:24:53.978 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.971+0000 7f9c899dc700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c84147970 con 0x7f9c840b23c0 2023-03-24T22:24:53.978 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.971+0000 7f9c827fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c840d1710 0x7f9c84148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.978 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.971+0000 7f9c827fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c840d1710 0x7f9c84148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43960/0 (socket says 172.21.15.18:43960) 2023-03-24T22:24:53.978 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.971+0000 7f9c827fc700 1 -- 172.21.15.18:0/3066870904 learned_addr learned my addr 172.21.15.18:0/3066870904 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:53.978 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c837fe700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c840cb2b0 0x7f9c840cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.978 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c82ffd700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 0x7f9c841472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.979 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c827fc700 1 -- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 msgr2=0x7f9c841472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.979 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c827fc700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 0x7f9c841472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.979 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c827fc700 1 -- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c840cb2b0 msgr2=0x7f9c840cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:53.979 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c827fc700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c840cb2b0 0x7f9c840cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:53.979 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c827fc700 1 -- 172.21.15.18:0/3066870904 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c740094a0 con 0x7f9c840d1710 2023-03-24T22:24:53.979 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c82ffd700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 0x7f9c841472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-24T22:24:53.979 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c827fc700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c840d1710 0x7f9c84148a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9c740090b0 tx=0x7f9c74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c6bfff700 1 -- 172.21.15.18:0/3066870904 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c740036a0 con 0x7f9c840d1710 2023-03-24T22:24:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c6bfff700 1 -- 172.21.15.18:0/3066870904 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7401f070 con 0x7f9c840d1710 2023-03-24T22:24:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c899dc700 1 -- 172.21.15.18:0/3066870904 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c840d2a80 con 0x7f9c840d1710 2023-03-24T22:24:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c6bfff700 1 -- 172.21.15.18:0/3066870904 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74003a40 con 0x7f9c840d1710 2023-03-24T22:24:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.972+0000 7f9c899dc700 1 -- 172.21.15.18:0/3066870904 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c840cdc00 con 0x7f9c840d1710 2023-03-24T22:24:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.973+0000 7f9c899dc700 1 -- 172.21.15.18:0/3066870904 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c840cf530 con 0x7f9c840d1710 2023-03-24T22:24:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.973+0000 7f9c6bfff700 1 -- 172.21.15.18:0/3066870904 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f9c74003ba0 con 0x7f9c840d1710 2023-03-24T22:24:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.974+0000 7f9c6bfff700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f9c70069de0 0x7f9c7006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.974+0000 7f9c6bfff700 1 -- 172.21.15.18:0/3066870904 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9c74021070 con 0x7f9c840d1710 2023-03-24T22:24:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.974+0000 7f9c82ffd700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f9c70069de0 0x7f9c7006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.974+0000 7f9c82ffd700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f9c70069de0 0x7f9c7006c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009ac0 tx=0x7f9c78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:53.978+0000 7f9c6bfff700 1 -- 172.21.15.18:0/3066870904 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9c74055270 con 0x7f9c840d1710 2023-03-24T22:24:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.179+0000 7f9c899dc700 1 -- 172.21.15.18:0/3066870904 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c84148860 con 0x7f9c840d1710 2023-03-24T22:24:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.180+0000 7f9c6bfff700 1 -- 172.21.15.18:0/3066870904 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14207 (secure 0 0 0) 0x7f9c74058890 con 0x7f9c840d1710 2023-03-24T22:24:54.181 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:54.181 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":17,"fsid":"67a1ac4c-1568-462e-b9b7-d22a9a29a86a","created":"2023-03-24T22:24:39.069846+0000","modified":"2023-03-24T22:24:53.319230+0000","last_up_change":"2023-03-24T22:24:49.299255+0000","last_in_change":"2023-03-24T22:24:41.559093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-03-24T22:24:50.404940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-03-24T22:24:50.798960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d184f8d0-aaf7-421c-9c33-c9f32a0e9236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6819","nonce":114916},{"type":"v1","addr":"172.21.15.18:6821","nonce":114916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6823","nonce":114916},{"type":"v1","addr":"172.21.15.18:6825","nonce":114916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":114916},{"type":"v1","addr":"172.21.15.18:6831","nonce":114916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6827","nonce":114916},{"type":"v1","addr":"172.21.15.18:6829","nonce":114916}]},"public_addr":"172.21.15.18:6821/114916","cluster_addr":"172.21.15.18:6825/114916","heartbeat_back_addr":"172.21.15.18:6831/114916","heartbeat_front_addr":"172.21.15.18:6829/114916","state":["exists","up"]},{"osd":1,"uuid":"d46c78b3-52ad-41f3-8c64-3d37e9adee63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":114913},{"type":"v1","addr":"172.21.15.18:6801","nonce":114913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":114913},{"type":"v1","addr":"172.21.15.18:6803","nonce":114913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":114913},{"type":"v1","addr":"172.21.15.18:6807","nonce":114913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":114913},{"type":"v1","addr":"172.21.15.18:6805","nonce":114913}]},"public_addr":"172.21.15.18:6801/114913","cluster_addr":"172.21.15.18:6803/114913","heartbeat_back_addr":"172.21.15.18:6807/114913","heartbeat_front_addr":"172.21.15.18:6805/114913","state":["exists","up"]},{"osd":2,"uuid":"4e4e783f-f720-42f9-b394-002a79f5eb0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":114914},{"type":"v1","addr":"172.21.15.18:6809","nonce":114914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":114914},{"type":"v1","addr":"172.21.15.18:6811","nonce":114914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":114914},{"type":"v1","addr":"172.21.15.18:6815","nonce":114914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":114914},{"type":"v1","addr":"172.21.15.18:6813","nonce":114914}]},"public_addr":"172.21.15.18:6809/114914","cluster_addr":"172.21.15.18:6811/114914","heartbeat_back_addr":"172.21.15.18:6815/114914","heartbeat_front_addr":"172.21.15.18:6813/114914","state":["exists","up"]},{"osd":3,"uuid":"2a8e5486-0ae6-43c3-870e-07ad88a43340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":114915},{"type":"v1","addr":"172.21.15.18:6817","nonce":114915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":114915},{"type":"v1","addr":"172.21.15.18:6820","nonce":114915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":114915},{"type":"v1","addr":"172.21.15.18:6828","nonce":114915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":114915},{"type":"v1","addr":"172.21.15.18:6824","nonce":114915}]},"public_addr":"172.21.15.18:6817/114915","cluster_addr":"172.21.15.18:6820/114915","heartbeat_back_addr":"172.21.15.18:6828/114915","heartbeat_front_addr":"172.21.15.18:6824/114915","state":["exists","up"]},{"osd":4,"uuid":"9a82df25-c818-47d7-8a9e-3e6fd55cd4f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":115214},{"type":"v1","addr":"172.21.15.100:6801","nonce":115214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":115214},{"type":"v1","addr":"172.21.15.100:6803","nonce":115214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":115214},{"type":"v1","addr":"172.21.15.100:6807","nonce":115214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":115214},{"type":"v1","addr":"172.21.15.100:6805","nonce":115214}]},"public_addr":"172.21.15.100:6801/115214","cluster_addr":"172.21.15.100:6803/115214","heartbeat_back_addr":"172.21.15.100:6807/115214","heartbeat_front_addr":"172.21.15.100:6805/115214","state":["exists","up"]},{"osd":5,"uuid":"ae37e2cd-d079-4962-9bd7-628aa5a7e14d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":115215},{"type":"v1","addr":"172.21.15.100:6809","nonce":115215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":115215},{"type":"v1","addr":"172.21.15.100:6811","nonce":115215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":115215},{"type":"v1","addr":"172.21.15.100:6815","nonce":115215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":115215},{"type":"v1","addr":"172.21.15.100:6813","nonce":115215}]},"public_addr":"172.21.15.100:6809/115215","cluster_addr":"172.21.15.100:6811/115215","heartbeat_back_addr":"172.21.15.100:6815/115215","heartbeat_front_addr":"172.21.15.100:6813/115215","state":["exists","up"]},{"osd":6,"uuid":"e3574199-2966-431b-ab0b-25efed52c749","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":115255},{"type":"v1","addr":"172.21.15.100:6825","nonce":115255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":115255},{"type":"v1","addr":"172.21.15.100:6827","nonce":115255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":115255},{"type":"v1","addr":"172.21.15.100:6831","nonce":115255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":115255},{"type":"v1","addr":"172.21.15.100:6829","nonce":115255}]},"public_addr":"172.21.15.100:6825/115255","cluster_addr":"172.21.15.100:6827/115255","heartbeat_back_addr":"172.21.15.100:6831/115255","heartbeat_front_addr":"172.21.15.100:6829/115255","state":["exists","up"]},{"osd":7,"uuid":"5a2a90ec-837f-4560-8a8e-b3e82eab590b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":115216},{"type":"v1","addr":"172.21.15.100:6817","nonce":115216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":115216},{"type":"v1","addr":"172.21.15.100:6819","nonce":115216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":115216},{"type":"v1","addr":"172.21.15.100:6823","nonce":115216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":115216},{"type":"v1","addr":"172.21.15.100:6821","nonce":115216}]},"public_addr":"172.21.15.100:6817/115216","cluster_addr":"172.21.15.100:6819/115216","heartbeat_back_addr":"172.21.15.100:6823/115216","heartbeat_front_addr":"172.21.15.100:6821/115216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.718826+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.668044+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.678787+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.750169+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.685071+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.741353+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.757517+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-24T22:24:46.737387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-24T22:24:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 -- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f9c70069de0 msgr2=0x7f9c7006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f9c70069de0 0x7f9c7006c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009ac0 tx=0x7f9c78008040 comp rx=0 tx=0).stop 2023-03-24T22:24:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 -- 172.21.15.18:0/3066870904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c840d1710 msgr2=0x7f9c84148a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c840d1710 0x7f9c84148a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9c740090b0 tx=0x7f9c74004750 comp rx=0 tx=0).stop 2023-03-24T22:24:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 -- 172.21.15.18:0/3066870904 shutdown_connections 2023-03-24T22:24:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f9c70069de0 0x7f9c7006c290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c840cb2b0 0x7f9c840cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840b23c0 0x7f9c841472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 --2- 172.21.15.18:0/3066870904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c840d1710 0x7f9c84148a10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 -- 172.21.15.18:0/3066870904 >> 172.21.15.18:0/3066870904 conn(0x7f9c84066140 msgr2=0x7f9c840b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 -- 172.21.15.18:0/3066870904 shutdown_connections 2023-03-24T22:24:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.182+0000 7f9c899dc700 1 -- 172.21.15.18:0/3066870904 wait complete. 2023-03-24T22:24:54.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-03-24T22:24:50.404940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-03-24T22:24:50.798960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-24T22:24:54.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-24T22:24:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.323+0000 7f3468bb8700 1 Processor -- start 2023-03-24T22:24:54.324 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.323+0000 7f3468bb8700 1 -- start start 2023-03-24T22:24:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3468bb8700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f34640cdc80 0x7f34640d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3468bb8700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3464070230 con 0x7f34640ce050 2023-03-24T22:24:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3468bb8700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346406fc70 con 0x7f34640afad0 2023-03-24T22:24:54.327 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3468bb8700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34640d0d80 con 0x7f34640cdc80 2023-03-24T22:24:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f345bfff700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f34640ce050 0x7f3464160fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38256/0 (socket says 172.21.15.18:38256) 2023-03-24T22:24:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f345bfff700 1 -- 172.21.15.18:0/3294261128 learned_addr learned my addr 172.21.15.18:0/3294261128 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f346259c700 1 --2- 172.21.15.18:0/3294261128 >> v2:172.21.15.100:3300/0 conn(0x7f34640cdc80 0x7f34640d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533803163 0 0) 0x7f3464070230 con 0x7f34640ce050 2023-03-24T22:24:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34500035b0 con 0x7f34640ce050 2023-03-24T22:24:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 <== mon.0 v1:172.21.15.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 784875785 0 0) 0x7f34500035b0 con 0x7f34640ce050 2023-03-24T22:24:54.328 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 >> v2:172.21.15.100:3300/0 conn(0x7f34640cdc80 msgr2=0x7f34640d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:54.329 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3461d9b700 1 --2- 172.21.15.18:0/3294261128 >> v2:172.21.15.100:3300/0 conn(0x7f34640cdc80 0x7f34640d0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.329 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 >> v1:172.21.15.100:6789/0 conn(0x7f34640afad0 legacy=0x7f346406c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.329 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 --> v1:172.21.15.18:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346406ff50 con 0x7f34640ce050 2023-03-24T22:24:54.329 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.324+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 <== mon.0 v1:172.21.15.18:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3658844337 0 0) 0x7f344c001670 con 0x7f34640ce050 2023-03-24T22:24:54.329 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 >> v1:172.21.15.18:6789/0 conn(0x7f34640ce050 legacy=0x7f3464160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.329 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3461d9b700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3450003a00 0x7f3450003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.329 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3461d9b700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34500043f0 0x7f34500048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.330 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3461d9b700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f345000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.330 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34640d0d80 con 0x7f34500050b0 2023-03-24T22:24:54.330 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345000c000 con 0x7f3450003a00 2023-03-24T22:24:54.330 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345000c1c0 con 0x7f34500043f0 2023-03-24T22:24:54.330 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3462d9d700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f345000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.330 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f346259c700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3450003a00 0x7f3450003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.330 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f345bfff700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34500043f0 0x7f34500048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.331 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3462d9d700 1 -- 172.21.15.18:0/3294261128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34500043f0 msgr2=0x7f34500048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:54.331 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3462d9d700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34500043f0 0x7f34500048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.331 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3462d9d700 1 -- 172.21.15.18:0/3294261128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3450003a00 msgr2=0x7f3450003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:54.331 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3462d9d700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3450003a00 0x7f3450003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.331 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3462d9d700 1 -- 172.21.15.18:0/3294261128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500035b0 con 0x7f34500050b0 2023-03-24T22:24:54.331 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3462d9d700 1 -- 172.21.15.18:0/3294261128 reap_dead start 2023-03-24T22:24:54.331 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3462d9d700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f345000ba10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f345c00c290 tx=0x7f3450005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779fd0ae5bfd60e4 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:54.332 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c001e70 con 0x7f34500050b0 2023-03-24T22:24:54.332 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345c00ce70 con 0x7f34500050b0 2023-03-24T22:24:54.332 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.325+0000 7f3461d9b700 1 -- 172.21.15.18:0/3294261128 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c003c20 con 0x7f34500050b0 2023-03-24T22:24:54.332 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.326+0000 7f3468bb8700 1 -- 172.21.15.18:0/3294261128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 msgr2=0x7f345000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.332 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.326+0000 7f3468bb8700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f345000ba10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f345c00c290 tx=0x7f3450005880 comp rx=0 tx=0).stop 2023-03-24T22:24:54.332 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.326+0000 7f3468bb8700 1 -- 172.21.15.18:0/3294261128 shutdown_connections 2023-03-24T22:24:54.332 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.326+0000 7f3468bb8700 1 --2- 172.21.15.18:0/3294261128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f345000ba10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.332 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.326+0000 7f3468bb8700 1 -- 172.21.15.18:0/3294261128 >> 172.21.15.18:0/3294261128 conn(0x7f346406e360 msgr2=0x7f346406e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:54.333 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.326+0000 7f3468bb8700 1 -- 172.21.15.18:0/3294261128 shutdown_connections 2023-03-24T22:24:54.333 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.326+0000 7f3468bb8700 1 -- 172.21.15.18:0/3294261128 wait complete. 2023-03-24T22:24:54.335 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.327+0000 7f3468bb8700 1 Processor -- start 2023-03-24T22:24:54.335 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.327+0000 7f3468bb8700 1 -- start start 2023-03-24T22:24:54.335 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f3468bb8700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f34640c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.335 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f3468bb8700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3464150f50 0x7f346415ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.335 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f3468bb8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f346415f340 0x7f34640c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.336 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f3468bb8700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34640c6470 con 0x7f34500050b0 2023-03-24T22:24:54.336 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f3468bb8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346415f7e0 con 0x7f346415f340 2023-03-24T22:24:54.336 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f3468bb8700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34640c57f0 con 0x7f3464150f50 2023-03-24T22:24:54.336 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f346259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f34640c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.336 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f346259c700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f34640c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43986/0 (socket says 172.21.15.18:43986) 2023-03-24T22:24:54.336 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f346259c700 1 -- 172.21.15.18:0/3185537877 learned_addr learned my addr 172.21.15.18:0/3185537877 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:54.336 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f3462d9d700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f346415f340 0x7f34640c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.336 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f345bfff700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3464150f50 0x7f346415ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.337 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f346259c700 1 -- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3464150f50 msgr2=0x7f346415ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.337 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f346259c700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3464150f50 0x7f346415ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.337 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f346259c700 1 -- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f346415f340 msgr2=0x7f34640c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.337 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f346259c700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f346415f340 0x7f34640c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.337 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.328+0000 7f346259c700 1 -- 172.21.15.18:0/3185537877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3464160aa0 con 0x7f34500050b0 2023-03-24T22:24:54.337 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.329+0000 7f346259c700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f34640c6890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34540119d0 tx=0x7f3454011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:54.337 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.329+0000 7f345affd700 1 -- 172.21.15.18:0/3185537877 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400f8e0 con 0x7f34500050b0 2023-03-24T22:24:54.338 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.329+0000 7f345affd700 1 -- 172.21.15.18:0/3185537877 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345400fa40 con 0x7f34500050b0 2023-03-24T22:24:54.338 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.329+0000 7f3468bb8700 1 -- 172.21.15.18:0/3185537877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34641607c0 con 0x7f34500050b0 2023-03-24T22:24:54.338 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.329+0000 7f345affd700 1 -- 172.21.15.18:0/3185537877 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400fdd0 con 0x7f34500050b0 2023-03-24T22:24:54.338 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.329+0000 7f3468bb8700 1 -- 172.21.15.18:0/3185537877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34640d0840 con 0x7f34500050b0 2023-03-24T22:24:54.338 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.330+0000 7f3468bb8700 1 -- 172.21.15.18:0/3185537877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346404e850 con 0x7f34500050b0 2023-03-24T22:24:54.338 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.330+0000 7f345affd700 1 -- 172.21.15.18:0/3185537877 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3454014440 con 0x7f34500050b0 2023-03-24T22:24:54.338 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.331+0000 7f345affd700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3444069e10 0x7f344406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.338 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.331+0000 7f345affd700 1 -- 172.21.15.18:0/3185537877 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3454088310 con 0x7f34500050b0 2023-03-24T22:24:54.339 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.331+0000 7f345bfff700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3444069e10 0x7f344406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.339 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.331+0000 7f345bfff700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3444069e10 0x7f344406c2c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34640ad900 tx=0x7f344c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:54.339 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.337+0000 7f345affd700 1 -- 172.21.15.18:0/3185537877 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3454053eb0 con 0x7f34500050b0 2023-03-24T22:24:54.532 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.531+0000 7f3468bb8700 1 -- 172.21.15.18:0/3185537877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f34640b6e80 con 0x7f34500050b0 2023-03-24T22:24:54.532 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.531+0000 7f345affd700 1 -- 172.21.15.18:0/3185537877 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f34540574d0 con 0x7f34500050b0 2023-03-24T22:24:54.533 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 8 2023-03-24T22:24:54.534 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.533+0000 7f3468bb8700 1 -- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3444069e10 msgr2=0x7f344406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.535 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.533+0000 7f3468bb8700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3444069e10 0x7f344406c2c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34640ad900 tx=0x7f344c009040 comp rx=0 tx=0).stop 2023-03-24T22:24:54.535 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.533+0000 7f3468bb8700 1 -- 172.21.15.18:0/3185537877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 msgr2=0x7f34640c6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.535 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.533+0000 7f3468bb8700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f34640c6890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34540119d0 tx=0x7f3454011d90 comp rx=0 tx=0).stop 2023-03-24T22:24:54.535 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.533+0000 7f3468bb8700 1 -- 172.21.15.18:0/3185537877 shutdown_connections 2023-03-24T22:24:54.535 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.534+0000 7f3468bb8700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f3444069e10 0x7f344406c2c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.535 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.534+0000 7f3468bb8700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34500050b0 0x7f34640c6890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.536 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.534+0000 7f3468bb8700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f346415f340 0x7f34640c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.536 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.534+0000 7f3468bb8700 1 --2- 172.21.15.18:0/3185537877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3464150f50 0x7f346415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.536 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.534+0000 7f3468bb8700 1 -- 172.21.15.18:0/3185537877 >> 172.21.15.18:0/3185537877 conn(0x7f346406e360 msgr2=0x7f346406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:54.536 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.534+0000 7f3468bb8700 1 -- 172.21.15.18:0/3185537877 shutdown_connections 2023-03-24T22:24:54.536 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.534+0000 7f3468bb8700 1 -- 172.21.15.18:0/3185537877 wait complete. 2023-03-24T22:24:54.546 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-24T22:24:54.677 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.674+0000 7f5fcb3c3700 1 Processor -- start 2023-03-24T22:24:54.677 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.674+0000 7f5fcb3c3700 1 -- start start 2023-03-24T22:24:54.677 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fcb3c3700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc4160530 0x7f5fc406e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.677 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fcb3c3700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5fc406eec0 0x7f5fc4071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.677 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fcb3c3700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc415cf60 con 0x7f5fc415a6e0 2023-03-24T22:24:54.678 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fcb3c3700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc40cabe0 con 0x7f5fc406eec0 2023-03-24T22:24:54.678 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fcb3c3700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc41609a0 con 0x7f5fc4160530 2023-03-24T22:24:54.678 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fc915f700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f5fc415a6e0 0x7f5fc40ca4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38258/0 (socket says 172.21.15.18:38258) 2023-03-24T22:24:54.678 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fc915f700 1 -- 172.21.15.18:0/1192056391 learned_addr learned my addr 172.21.15.18:0/1192056391 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:54.678 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fc895e700 1 --2- 172.21.15.18:0/1192056391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc4160530 0x7f5fc406e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.678 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fc9960700 1 --2- 172.21.15.18:0/1192056391 >> v2:172.21.15.100:3300/0 conn(0x7f5fc406eec0 0x7f5fc4071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.678 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fbbfff700 1 -- 172.21.15.18:0/1192056391 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971658988 0 0) 0x7f5fc415cf60 con 0x7f5fc415a6e0 2023-03-24T22:24:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.675+0000 7f5fbbfff700 1 -- 172.21.15.18:0/1192056391 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb00035b0 con 0x7f5fc415a6e0 2023-03-24T22:24:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fbbfff700 1 -- 172.21.15.18:0/1192056391 <== mon.0 v1:172.21.15.18:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 26910447 0 0) 0x7f5fb00035b0 con 0x7f5fc415a6e0 2023-03-24T22:24:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fc9960700 1 -- 172.21.15.18:0/1192056391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc4160530 msgr2=0x7f5fc406e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fc9960700 1 --2- 172.21.15.18:0/1192056391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc4160530 0x7f5fc406e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fc9960700 1 -- 172.21.15.18:0/1192056391 >> v1:172.21.15.18:6789/0 conn(0x7f5fc415a6e0 legacy=0x7f5fc40ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fc9960700 1 -- 172.21.15.18:0/1192056391 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc4160c80 con 0x7f5fc406eec0 2023-03-24T22:24:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fc895e700 1 --2- 172.21.15.18:0/1192056391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc4160530 0x7f5fc406e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:54.679 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fc9960700 1 --2- 172.21.15.18:0/1192056391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc406eec0 0x7f5fc4071130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009d90 tx=0x7f5fbc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2010023af5c53898 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:54.680 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fbbfff700 1 -- 172.21.15.18:0/1192056391 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc00e070 con 0x7f5fc406eec0 2023-03-24T22:24:54.680 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fbbfff700 1 -- 172.21.15.18:0/1192056391 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fbc003a20 con 0x7f5fc406eec0 2023-03-24T22:24:54.680 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fbbfff700 1 -- 172.21.15.18:0/1192056391 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4001670 con 0x7f5fc406eec0 2023-03-24T22:24:54.682 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/1192056391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc406eec0 msgr2=0x7f5fc4071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.682 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fcb3c3700 1 --2- 172.21.15.18:0/1192056391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc406eec0 0x7f5fc4071130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009d90 tx=0x7f5fbc00b330 comp rx=0 tx=0).stop 2023-03-24T22:24:54.682 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/1192056391 shutdown_connections 2023-03-24T22:24:54.683 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fcb3c3700 1 --2- 172.21.15.18:0/1192056391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc406eec0 0x7f5fc4071130 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.683 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fcb3c3700 1 --2- 172.21.15.18:0/1192056391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc4160530 0x7f5fc406e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.683 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.676+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/1192056391 >> 172.21.15.18:0/1192056391 conn(0x7f5fc40cd590 msgr2=0x7f5fc40cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:54.683 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.677+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/1192056391 shutdown_connections 2023-03-24T22:24:54.683 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.677+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/1192056391 wait complete. 2023-03-24T22:24:54.683 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.678+0000 7f5fcb3c3700 1 Processor -- start 2023-03-24T22:24:54.683 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.678+0000 7f5fcb3c3700 1 -- start start 2023-03-24T22:24:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.678+0000 7f5fcb3c3700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc406eec0 0x7f5fc41563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fcb3c3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fc415a6e0 0x7f5fc40b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fcb3c3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4160530 0x7f5fc40b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fcb3c3700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc4156930 con 0x7f5fc415a6e0 2023-03-24T22:24:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fcb3c3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40b4730 con 0x7f5fc4160530 2023-03-24T22:24:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fcb3c3700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40b5270 con 0x7f5fc406eec0 2023-03-24T22:24:54.684 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc895e700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fc415a6e0 0x7f5fc40b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.685 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc895e700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fc415a6e0 0x7f5fc40b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44002/0 (socket says 172.21.15.18:44002) 2023-03-24T22:24:54.685 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc895e700 1 -- 172.21.15.18:0/229708285 learned_addr learned my addr 172.21.15.18:0/229708285 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:54.685 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc9960700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4160530 0x7f5fc40b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.685 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc915f700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc406eec0 0x7f5fc41563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.685 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc895e700 1 -- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc406eec0 msgr2=0x7f5fc41563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.685 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc895e700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc406eec0 0x7f5fc41563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.685 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc895e700 1 -- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4160530 msgr2=0x7f5fc40b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.686 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc895e700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4160530 0x7f5fc40b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.686 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc895e700 1 -- 172.21.15.18:0/229708285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc00094a0 con 0x7f5fc415a6e0 2023-03-24T22:24:54.686 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc9960700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4160530 0x7f5fc40b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-24T22:24:54.686 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fc895e700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fc415a6e0 0x7f5fc40b2d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000ebe0 tx=0x7f5fc0009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:54.686 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fb9ffb700 1 -- 172.21.15.18:0/229708285 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000c8b0 con 0x7f5fc415a6e0 2023-03-24T22:24:54.686 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fb9ffb700 1 -- 172.21.15.18:0/229708285 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc000ca10 con 0x7f5fc415a6e0 2023-03-24T22:24:54.686 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/229708285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fbc009a70 con 0x7f5fc415a6e0 2023-03-24T22:24:54.686 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.679+0000 7f5fb9ffb700 1 -- 172.21.15.18:0/229708285 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000cda0 con 0x7f5fc415a6e0 2023-03-24T22:24:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.680+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/229708285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc4161240 con 0x7f5fc415a6e0 2023-03-24T22:24:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.680+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/229708285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc4160f60 con 0x7f5fc415a6e0 2023-03-24T22:24:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.681+0000 7f5fb9ffb700 1 -- 172.21.15.18:0/229708285 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f5fc0013470 con 0x7f5fc415a6e0 2023-03-24T22:24:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.681+0000 7f5fb9ffb700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f5fa8069de0 0x7f5fa806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.681+0000 7f5fb9ffb700 1 -- 172.21.15.18:0/229708285 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5fc00881f0 con 0x7f5fc415a6e0 2023-03-24T22:24:54.687 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.682+0000 7f5fc915f700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f5fa8069de0 0x7f5fa806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:54.688 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.682+0000 7f5fc915f700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f5fa8069de0 0x7f5fa806c290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40011e0 tx=0x7f5fb4001550 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:54.688 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.685+0000 7f5fb9ffb700 1 -- 172.21.15.18:0/229708285 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f5fc0053cf0 con 0x7f5fc415a6e0 2023-03-24T22:24:54.881 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.880+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/229708285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fc4159f20 con 0x7f5fc415a6e0 2023-03-24T22:24:54.881 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.880+0000 7f5fb9ffb700 1 -- 172.21.15.18:0/229708285 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fc0057310 con 0x7f5fc415a6e0 2023-03-24T22:24:54.881 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2023-03-24T22:24:54.883 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f5fa8069de0 msgr2=0x7f5fa806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.884 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f5fa8069de0 0x7f5fa806c290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40011e0 tx=0x7f5fb4001550 comp rx=0 tx=0).stop 2023-03-24T22:24:54.884 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/229708285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fc415a6e0 msgr2=0x7f5fc40b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:54.884 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fc415a6e0 0x7f5fc40b2d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000ebe0 tx=0x7f5fc0009e20 comp rx=0 tx=0).stop 2023-03-24T22:24:54.884 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/229708285 shutdown_connections 2023-03-24T22:24:54.884 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f5fa8069de0 0x7f5fa806c290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.884 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4160530 0x7f5fc40b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.884 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc406eec0 0x7f5fc41563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.885 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 --2- 172.21.15.18:0/229708285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fc415a6e0 0x7f5fc40b2d40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:54.885 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/229708285 >> 172.21.15.18:0/229708285 conn(0x7f5fc40cd590 msgr2=0x7f5fc415d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:54.885 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/229708285 shutdown_connections 2023-03-24T22:24:54.885 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:54.882+0000 7f5fcb3c3700 1 -- 172.21.15.18:0/229708285 wait complete. 2023-03-24T22:24:54.896 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-03-24T22:24:55.021 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.020+0000 7f1049855700 1 Processor -- start 2023-03-24T22:24:55.021 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.020+0000 7f1049855700 1 -- start start 2023-03-24T22:24:55.022 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f1049855700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f10440d01e0 0x7f10440d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.023 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f1049855700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10440d2830 con 0x7f10440b23c0 2023-03-24T22:24:55.023 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f1049855700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10440d2df0 con 0x7f10440b5f80 2023-03-24T22:24:55.023 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f1049855700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10440b2790 con 0x7f10440d01e0 2023-03-24T22:24:55.023 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f10437fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f10440d01e0 0x7f10440d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.023 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f1042ffd700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7f10440b23c0 0x7f10440d1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38268/0 (socket says 172.21.15.18:38268) 2023-03-24T22:24:55.023 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f10437fe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7f10440d01e0 0x7f10440d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44006/0 (socket says 172.21.15.18:44006) 2023-03-24T22:24:55.024 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f1042ffd700 1 -- 172.21.15.18:0/940036365 learned_addr learned my addr 172.21.15.18:0/940036365 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:55.024 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f10437fe700 1 -- 172.21.15.18:0/940036365 >> v1:172.21.15.100:6789/0 conn(0x7f10440b5f80 legacy=0x7f10440cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.024 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f10437fe700 1 -- 172.21.15.18:0/940036365 >> v1:172.21.15.18:6789/0 conn(0x7f10440b23c0 legacy=0x7f10440d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.024 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.021+0000 7f10437fe700 1 -- 172.21.15.18:0/940036365 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10440b6400 con 0x7f10440d01e0 2023-03-24T22:24:55.024 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f10437fe700 1 --2- 172.21.15.18:0/940036365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440d01e0 0x7f10440d3280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f103c009d90 tx=0x7f103c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cc7004986e6786 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:55.024 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f1041ffb700 1 -- 172.21.15.18:0/940036365 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c00e070 con 0x7f10440d01e0 2023-03-24T22:24:55.024 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f1041ffb700 1 -- 172.21.15.18:0/940036365 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c00bbd0 con 0x7f10440d01e0 2023-03-24T22:24:55.024 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f1041ffb700 1 -- 172.21.15.18:0/940036365 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c012430 con 0x7f10440d01e0 2023-03-24T22:24:55.026 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f1049855700 1 -- 172.21.15.18:0/940036365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440d01e0 msgr2=0x7f10440d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.027 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f1049855700 1 --2- 172.21.15.18:0/940036365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440d01e0 0x7f10440d3280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f103c009d90 tx=0x7f103c00b330 comp rx=0 tx=0).stop 2023-03-24T22:24:55.027 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f1049855700 1 -- 172.21.15.18:0/940036365 shutdown_connections 2023-03-24T22:24:55.027 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f1049855700 1 --2- 172.21.15.18:0/940036365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440d01e0 0x7f10440d3280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.027 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f1049855700 1 -- 172.21.15.18:0/940036365 >> 172.21.15.18:0/940036365 conn(0x7f1044066140 msgr2=0x7f1044066540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:55.027 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f1049855700 1 -- 172.21.15.18:0/940036365 shutdown_connections 2023-03-24T22:24:55.027 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.022+0000 7f1049855700 1 -- 172.21.15.18:0/940036365 wait complete. 2023-03-24T22:24:55.028 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.023+0000 7f1049855700 1 Processor -- start 2023-03-24T22:24:55.028 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.024+0000 7f1049855700 1 -- start start 2023-03-24T22:24:55.028 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.024+0000 7f1049855700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10440b23c0 0x7f10441e6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.028 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.024+0000 7f1049855700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440b5f80 0x7f10441519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.028 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.024+0000 7f1049855700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10440d01e0 0x7f10440c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.028 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.024+0000 7f1049855700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10440c71b0 con 0x7f10440b5f80 2023-03-24T22:24:55.028 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.024+0000 7f1049855700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10440c6ba0 con 0x7f10440d01e0 2023-03-24T22:24:55.028 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.024+0000 7f1049855700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10440cc490 con 0x7f10440b23c0 2023-03-24T22:24:55.029 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.024+0000 7f10427fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440b5f80 0x7f10441519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.029 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f10427fc700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440b5f80 0x7f10441519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44014/0 (socket says 172.21.15.18:44014) 2023-03-24T22:24:55.029 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f10427fc700 1 -- 172.21.15.18:0/2398294769 learned_addr learned my addr 172.21.15.18:0/2398294769 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:55.029 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f10437fe700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10440d01e0 0x7f10440c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.029 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f1042ffd700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10440b23c0 0x7f10441e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.029 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f10427fc700 1 -- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10440b23c0 msgr2=0x7f10441e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.029 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f10427fc700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10440b23c0 0x7f10441e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.030 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f10427fc700 1 -- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10440d01e0 msgr2=0x7f10440c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.030 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f10427fc700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10440d01e0 0x7f10440c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.030 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f10427fc700 1 -- 172.21.15.18:0/2398294769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c009a70 con 0x7f10440b5f80 2023-03-24T22:24:55.030 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f1042ffd700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10440b23c0 0x7f10441e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-24T22:24:55.030 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f10427fc700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440b5f80 0x7f10441519d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f103400c4e0 tx=0x7f103400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:55.030 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f102bfff700 1 -- 172.21.15.18:0/2398294769 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10340052d0 con 0x7f10440b5f80 2023-03-24T22:24:55.030 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f102bfff700 1 -- 172.21.15.18:0/2398294769 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1034005430 con 0x7f10440b5f80 2023-03-24T22:24:55.030 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f1049855700 1 -- 172.21.15.18:0/2398294769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10440cdc00 con 0x7f10440b5f80 2023-03-24T22:24:55.031 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f102bfff700 1 -- 172.21.15.18:0/2398294769 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10340038c0 con 0x7f10440b5f80 2023-03-24T22:24:55.031 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.025+0000 7f1049855700 1 -- 172.21.15.18:0/2398294769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10440cd920 con 0x7f10440b5f80 2023-03-24T22:24:55.031 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.026+0000 7f1049855700 1 -- 172.21.15.18:0/2398294769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10440d2a80 con 0x7f10440b5f80 2023-03-24T22:24:55.032 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.027+0000 7f102bfff700 1 -- 172.21.15.18:0/2398294769 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1034003a20 con 0x7f10440b5f80 2023-03-24T22:24:55.032 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.027+0000 7f102bfff700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1030069de0 0x7f103006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.032 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.027+0000 7f102bfff700 1 -- 172.21.15.18:0/2398294769 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1034087100 con 0x7f10440b5f80 2023-03-24T22:24:55.032 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.027+0000 7f1042ffd700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1030069de0 0x7f103006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.032 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.027+0000 7f1042ffd700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1030069de0 0x7f103006c290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1038005910 tx=0x7f1038009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:55.033 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.031+0000 7f102bfff700 1 -- 172.21.15.18:0/2398294769 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1034052d90 con 0x7f10440b5f80 2023-03-24T22:24:55.231 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.230+0000 7f1049855700 1 -- 172.21.15.18:0/2398294769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f104404e850 con 0x7f10440b5f80 2023-03-24T22:24:55.231 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.230+0000 7f102bfff700 1 -- 172.21.15.18:0/2398294769 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f10340563b0 con 0x7f10440b5f80 2023-03-24T22:24:55.231 INFO:teuthology.orchestra.run.smithi018.stdout: 2023-03-24T22:24:55.232 INFO:teuthology.orchestra.run.smithi018.stdout:[ 2023-03-24T22:24:55.232 INFO:teuthology.orchestra.run.smithi018.stdout: { 2023-03-24T22:24:55.232 INFO:teuthology.orchestra.run.smithi018.stdout: "poolnum": 1, 2023-03-24T22:24:55.232 INFO:teuthology.orchestra.run.smithi018.stdout: "poolname": "rbd" 2023-03-24T22:24:55.232 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2023-03-24T22:24:55.232 INFO:teuthology.orchestra.run.smithi018.stdout: { 2023-03-24T22:24:55.232 INFO:teuthology.orchestra.run.smithi018.stdout: "poolnum": 2, 2023-03-24T22:24:55.233 INFO:teuthology.orchestra.run.smithi018.stdout: "poolname": ".mgr" 2023-03-24T22:24:55.233 INFO:teuthology.orchestra.run.smithi018.stdout: } 2023-03-24T22:24:55.233 INFO:teuthology.orchestra.run.smithi018.stdout:] 2023-03-24T22:24:55.234 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 -- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1030069de0 msgr2=0x7f103006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.234 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1030069de0 0x7f103006c290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1038005910 tx=0x7f1038009040 comp rx=0 tx=0).stop 2023-03-24T22:24:55.234 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 -- 172.21.15.18:0/2398294769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440b5f80 msgr2=0x7f10441519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.235 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440b5f80 0x7f10441519d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f103400c4e0 tx=0x7f103400c8a0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.235 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 -- 172.21.15.18:0/2398294769 shutdown_connections 2023-03-24T22:24:55.235 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f1030069de0 0x7f103006c290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10440d01e0 0x7f10440c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10440b23c0 0x7f10441e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 --2- 172.21.15.18:0/2398294769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f10440b5f80 0x7f10441519d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.236 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 -- 172.21.15.18:0/2398294769 >> 172.21.15.18:0/2398294769 conn(0x7f1044066140 msgr2=0x7f10440b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:55.237 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 -- 172.21.15.18:0/2398294769 shutdown_connections 2023-03-24T22:24:55.237 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.232+0000 7f1049855700 1 -- 172.21.15.18:0/2398294769 wait complete. 2023-03-24T22:24:55.245 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-03-24T22:24:55.246 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-03-24T22:24:55.376 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.375+0000 7f59d2ee3700 1 Processor -- start 2023-03-24T22:24:55.376 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.375+0000 7f59d2ee3700 1 -- start start 2023-03-24T22:24:55.377 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.376+0000 7f59d2ee3700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f59cc070810 0x7f59cc070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.377 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.376+0000 7f59d2ee3700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc0713b0 0x7f59cc06a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.377 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.376+0000 7f59d2ee3700 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc1595b0 con 0x7f59cc15d680 2023-03-24T22:24:55.378 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.376+0000 7f59d2ee3700 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc15db40 con 0x7f59cc070810 2023-03-24T22:24:55.378 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.376+0000 7f59d2ee3700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc071120 con 0x7f59cc0713b0 2023-03-24T22:24:55.378 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.376+0000 7f59cbfff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc0713b0 0x7f59cc06a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.378 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.376+0000 7f59d0c7f700 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f59cc070810 0x7f59cc070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.378 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.376+0000 7f59d1480700 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f59cc15d680 0x7f59cc06a9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.18:49374/0 (socket says 172.21.15.18:49374) 2023-03-24T22:24:55.379 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.376+0000 7f59d1480700 1 -- 172.21.15.18:0/583767936 learned_addr learned my addr 172.21.15.18:0/583767936 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:55.379 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.376+0000 7f59cbfff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc0713b0 0x7f59cc06a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.18:34162/0 (socket says 172.21.15.18:34162) 2023-03-24T22:24:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.377+0000 7f59d0c7f700 1 -- 172.21.15.18:0/583767936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc0713b0 msgr2=0x7f59cc06a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.377+0000 7f59d0c7f700 1 --2- 172.21.15.18:0/583767936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc0713b0 0x7f59cc06a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.377+0000 7f59d0c7f700 1 -- 172.21.15.18:0/583767936 >> v1:172.21.15.100:6789/0 conn(0x7f59cc15d680 legacy=0x7f59cc06a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.377+0000 7f59d0c7f700 1 -- 172.21.15.18:0/583767936 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc0ca480 con 0x7f59cc070810 2023-03-24T22:24:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.377+0000 7f59cb7fe700 1 -- 172.21.15.18:0/583767936 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404815350 0 0) 0x7f59cc1595b0 con 0x7f59cc15d680 2023-03-24T22:24:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.377+0000 7f59cbfff700 1 --2- 172.21.15.18:0/583767936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc0713b0 0x7f59cc06a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-24T22:24:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.377+0000 7f59d0c7f700 1 --2- 172.21.15.18:0/583767936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59cc070810 0x7f59cc070be0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009840 tx=0x7f59c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f32bf2dfc823c23 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.377+0000 7f59cb7fe700 1 -- 172.21.15.18:0/583767936 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0011830 con 0x7f59cc070810 2023-03-24T22:24:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.377+0000 7f59cb7fe700 1 -- 172.21.15.18:0/583767936 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c0011990 con 0x7f59cc070810 2023-03-24T22:24:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.377+0000 7f59cb7fe700 1 -- 172.21.15.18:0/583767936 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0011d20 con 0x7f59cc070810 2023-03-24T22:24:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.378+0000 7f59d2ee3700 1 -- 172.21.15.18:0/583767936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59cc070810 msgr2=0x7f59cc070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.378+0000 7f59d2ee3700 1 --2- 172.21.15.18:0/583767936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59cc070810 0x7f59cc070be0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009840 tx=0x7f59c0009b50 comp rx=0 tx=0).stop 2023-03-24T22:24:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.378+0000 7f59d2ee3700 1 -- 172.21.15.18:0/583767936 shutdown_connections 2023-03-24T22:24:55.384 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.378+0000 7f59d2ee3700 1 --2- 172.21.15.18:0/583767936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59cc070810 0x7f59cc070be0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.384 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.378+0000 7f59d2ee3700 1 --2- 172.21.15.18:0/583767936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc0713b0 0x7f59cc06a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.384 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.378+0000 7f59d2ee3700 1 -- 172.21.15.18:0/583767936 >> 172.21.15.18:0/583767936 conn(0x7f59cc06d5a0 msgr2=0x7f59cc0cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:55.384 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.378+0000 7f59d2ee3700 1 -- 172.21.15.18:0/583767936 shutdown_connections 2023-03-24T22:24:55.384 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.378+0000 7f59d2ee3700 1 -- 172.21.15.18:0/583767936 wait complete. 2023-03-24T22:24:55.384 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.379+0000 7f59d2ee3700 1 Processor -- start 2023-03-24T22:24:55.384 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.379+0000 7f59d2ee3700 1 -- start start 2023-03-24T22:24:55.385 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d2ee3700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc070810 0x7f59cc165250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.385 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d2ee3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59cc0713b0 0x7f59cc15b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.385 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d2ee3700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59cc15d680 0x7f59cc0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.385 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d2ee3700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc15af80 con 0x7f59cc15d680 2023-03-24T22:24:55.385 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d2ee3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc0b3bc0 con 0x7f59cc0713b0 2023-03-24T22:24:55.385 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d2ee3700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc0b3ea0 con 0x7f59cc070810 2023-03-24T22:24:55.385 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d1480700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59cc15d680 0x7f59cc0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.386 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d1480700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59cc15d680 0x7f59cc0b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44028/0 (socket says 172.21.15.18:44028) 2023-03-24T22:24:55.386 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d1480700 1 -- 172.21.15.18:0/3463470678 learned_addr learned my addr 172.21.15.18:0/3463470678 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:55.386 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d0c7f700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc070810 0x7f59cc165250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.386 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59cbfff700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59cc0713b0 0x7f59cc15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.386 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d1480700 1 -- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc070810 msgr2=0x7f59cc165250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d1480700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc070810 0x7f59cc165250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d1480700 1 -- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59cc0713b0 msgr2=0x7f59cc15b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d1480700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59cc0713b0 0x7f59cc15b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d1480700 1 -- 172.21.15.18:0/3463470678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b80094a0 con 0x7f59cc15d680 2023-03-24T22:24:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.380+0000 7f59d1480700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59cc15d680 0x7f59cc0b3050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59c400cc10 tx=0x7f59c400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.381+0000 7f59c97fa700 1 -- 172.21.15.18:0/3463470678 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c400aaa0 con 0x7f59cc15d680 2023-03-24T22:24:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.381+0000 7f59c97fa700 1 -- 172.21.15.18:0/3463470678 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c400ac20 con 0x7f59cc15d680 2023-03-24T22:24:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.381+0000 7f59d2ee3700 1 -- 172.21.15.18:0/3463470678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c00094a0 con 0x7f59cc15d680 2023-03-24T22:24:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.381+0000 7f59c97fa700 1 -- 172.21.15.18:0/3463470678 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40070d0 con 0x7f59cc15d680 2023-03-24T22:24:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.381+0000 7f59d2ee3700 1 -- 172.21.15.18:0/3463470678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc0caa40 con 0x7f59cc15d680 2023-03-24T22:24:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.381+0000 7f59d2ee3700 1 -- 172.21.15.18:0/3463470678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc0ca760 con 0x7f59cc15d680 2023-03-24T22:24:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.382+0000 7f59c97fa700 1 -- 172.21.15.18:0/3463470678 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f59c4007230 con 0x7f59cc15d680 2023-03-24T22:24:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.382+0000 7f59c97fa700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f59b0069de0 0x7f59b006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.382+0000 7f59c97fa700 1 -- 172.21.15.18:0/3463470678 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f59c4087930 con 0x7f59cc15d680 2023-03-24T22:24:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.383+0000 7f59d0c7f700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f59b0069de0 0x7f59b006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.383+0000 7f59d0c7f700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f59b0069de0 0x7f59b006c290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f59c00090b0 tx=0x7f59c0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:55.390 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.386+0000 7f59c97fa700 1 -- 172.21.15.18:0/3463470678 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f59c4053530 con 0x7f59cc15d680 2023-03-24T22:24:55.583 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:55.582+0000 7f59d2ee3700 1 -- 172.21.15.18:0/3463470678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f59cc0cad20 con 0x7f59cc15d680 2023-03-24T22:24:56.347 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.346+0000 7f59c97fa700 1 -- 172.21.15.18:0/3463470678 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f59c4018020 con 0x7f59cc15d680 2023-03-24T22:24:56.349 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.348+0000 7f59d2ee3700 1 -- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f59b0069de0 msgr2=0x7f59b006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:56.350 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.348+0000 7f59d2ee3700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f59b0069de0 0x7f59b006c290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f59c00090b0 tx=0x7f59c0008030 comp rx=0 tx=0).stop 2023-03-24T22:24:56.350 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.348+0000 7f59d2ee3700 1 -- 172.21.15.18:0/3463470678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59cc15d680 msgr2=0x7f59cc0b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:56.350 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.348+0000 7f59d2ee3700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59cc15d680 0x7f59cc0b3050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59c400cc10 tx=0x7f59c400cf20 comp rx=0 tx=0).stop 2023-03-24T22:24:56.351 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.348+0000 7f59d2ee3700 1 -- 172.21.15.18:0/3463470678 shutdown_connections 2023-03-24T22:24:56.351 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.348+0000 7f59d2ee3700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7f59b0069de0 0x7f59b006c290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:56.351 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.348+0000 7f59d2ee3700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59cc15d680 0x7f59cc0b3050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:56.351 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.348+0000 7f59d2ee3700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59cc070810 0x7f59cc165250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.348+0000 7f59d2ee3700 1 --2- 172.21.15.18:0/3463470678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59cc0713b0 0x7f59cc15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.348+0000 7f59d2ee3700 1 -- 172.21.15.18:0/3463470678 >> 172.21.15.18:0/3463470678 conn(0x7f59cc06d5a0 msgr2=0x7f59cc14f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.349+0000 7f59d2ee3700 1 -- 172.21.15.18:0/3463470678 shutdown_connections 2023-03-24T22:24:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.349+0000 7f59d2ee3700 1 -- 172.21.15.18:0/3463470678 wait complete. 2023-03-24T22:24:56.353 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'cephfs_metadata' created 2023-03-24T22:24:56.367 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-03-24T22:24:56.504 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.503+0000 7fa68bdfe700 1 Processor -- start 2023-03-24T22:24:56.504 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.503+0000 7fa68bdfe700 1 -- start start 2023-03-24T22:24:56.506 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.504+0000 7fa68bdfe700 1 --2- >> v2:172.21.15.18:3300/0 conn(0x7fa6840d01f0 0x7fa68406a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:56.506 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.504+0000 7fa68bdfe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa684070f90 0x7fa68406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:56.507 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.504+0000 7fa68bdfe700 1 -- --> v1:172.21.15.18:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6841614e0 con 0x7fa68415c9b0 2023-03-24T22:24:56.507 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.504+0000 7fa68bdfe700 1 -- --> v2:172.21.15.18:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68406bd20 con 0x7fa6840d01f0 2023-03-24T22:24:56.507 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.504+0000 7fa68bdfe700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6841605f0 con 0x7fa684070f90 2023-03-24T22:24:56.507 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.504+0000 7fa689b9a700 1 --1- >> v1:172.21.15.18:6789/0 conn(0x7fa68415c9b0 0x7fa684070880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.18:6789/0 says I am v1:172.21.15.18:38284/0 (socket says 172.21.15.18:38284) 2023-03-24T22:24:56.507 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.504+0000 7fa689b9a700 1 -- 172.21.15.18:0/4055048684 learned_addr learned my addr 172.21.15.18:0/4055048684 (peer_addr_for_me v1:172.21.15.18:0/0) 2023-03-24T22:24:56.507 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.504+0000 7fa689399700 1 --2- 172.21.15.18:0/4055048684 >> v2:172.21.15.18:3300/0 conn(0x7fa6840d01f0 0x7fa68406a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:56.507 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.504+0000 7fa68a39b700 1 --2- 172.21.15.18:0/4055048684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa684070f90 0x7fa68406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:56.508 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa688b98700 1 -- 172.21.15.18:0/4055048684 <== mon.0 v1:172.21.15.18:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391629225 0 0) 0x7fa6841614e0 con 0x7fa68415c9b0 2023-03-24T22:24:56.508 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa688b98700 1 -- 172.21.15.18:0/4055048684 --> v1:172.21.15.18:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6700035b0 con 0x7fa68415c9b0 2023-03-24T22:24:56.508 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa689399700 1 -- 172.21.15.18:0/4055048684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa684070f90 msgr2=0x7fa68406b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-24T22:24:56.508 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa689399700 1 --2- 172.21.15.18:0/4055048684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa684070f90 0x7fa68406b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:56.509 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa689399700 1 -- 172.21.15.18:0/4055048684 >> v1:172.21.15.18:6789/0 conn(0x7fa68415c9b0 legacy=0x7fa684070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:56.509 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa689399700 1 -- 172.21.15.18:0/4055048684 --> v2:172.21.15.18:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6841608d0 con 0x7fa6840d01f0 2023-03-24T22:24:56.509 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa689399700 1 --2- 172.21.15.18:0/4055048684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6840d01f0 0x7fa68406a4a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa674009840 tx=0x7fa674009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376c3623e4fcce8e server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:56.509 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa688b98700 1 -- 172.21.15.18:0/4055048684 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674011830 con 0x7fa6840d01f0 2023-03-24T22:24:56.509 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa688b98700 1 -- 172.21.15.18:0/4055048684 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa674011990 con 0x7fa6840d01f0 2023-03-24T22:24:56.509 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa688b98700 1 -- 172.21.15.18:0/4055048684 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674011d20 con 0x7fa6840d01f0 2023-03-24T22:24:56.509 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4055048684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6840d01f0 msgr2=0x7fa68406a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:56.510 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa68bdfe700 1 --2- 172.21.15.18:0/4055048684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6840d01f0 0x7fa68406a4a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa674009840 tx=0x7fa674009b50 comp rx=0 tx=0).stop 2023-03-24T22:24:56.510 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4055048684 shutdown_connections 2023-03-24T22:24:56.510 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa68bdfe700 1 --2- 172.21.15.18:0/4055048684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa684070f90 0x7fa68406b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:56.510 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.505+0000 7fa68bdfe700 1 --2- 172.21.15.18:0/4055048684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6840d01f0 0x7fa68406a4a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:56.510 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.506+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4055048684 >> 172.21.15.18:0/4055048684 conn(0x7fa6840ccd80 msgr2=0x7fa6840cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-24T22:24:56.510 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.506+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4055048684 shutdown_connections 2023-03-24T22:24:56.510 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.506+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4055048684 wait complete. 2023-03-24T22:24:56.512 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.507+0000 7fa68bdfe700 1 Processor -- start 2023-03-24T22:24:56.513 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.507+0000 7fa68bdfe700 1 -- start start 2023-03-24T22:24:56.513 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.507+0000 7fa68bdfe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa684070f90 0x7fa6841e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:56.513 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.507+0000 7fa68bdfe700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6840d01f0 0x7fa684159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:56.513 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.507+0000 7fa68bdfe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68415c9b0 0x7fa6840c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:56.513 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.507+0000 7fa68bdfe700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68415a450 con 0x7fa6840d01f0 2023-03-24T22:24:56.513 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.507+0000 7fa68bdfe700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6840c6740 con 0x7fa684070f90 2023-03-24T22:24:56.514 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa68bdfe700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa684150880 con 0x7fa68415c9b0 2023-03-24T22:24:56.514 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689399700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6840d01f0 0x7fa684159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:56.514 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689399700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6840d01f0 0x7fa684159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44038/0 (socket says 172.21.15.18:44038) 2023-03-24T22:24:56.514 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689399700 1 -- 172.21.15.18:0/4035454282 learned_addr learned my addr 172.21.15.18:0/4035454282 (peer_addr_for_me v2:172.21.15.18:0/0) 2023-03-24T22:24:56.514 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa68a39b700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68415c9b0 0x7fa6840c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:56.514 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689b9a700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa684070f90 0x7fa6841e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:56.514 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689399700 1 -- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68415c9b0 msgr2=0x7fa6840c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:56.515 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689399700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68415c9b0 0x7fa6840c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:56.515 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689399700 1 -- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa684070f90 msgr2=0x7fa6841e7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:56.515 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689399700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa684070f90 0x7fa6841e7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:56.515 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689399700 1 -- 172.21.15.18:0/4035454282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740094a0 con 0x7fa6840d01f0 2023-03-24T22:24:56.515 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689b9a700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa684070f90 0x7fa6841e7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-24T22:24:56.515 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa689399700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6840d01f0 0x7fa684159f10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa674000c00 tx=0x7fa674004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:56.515 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa67a7fc700 1 -- 172.21.15.18:0/4035454282 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6740036a0 con 0x7fa6840d01f0 2023-03-24T22:24:56.515 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa67a7fc700 1 -- 172.21.15.18:0/4035454282 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67401f070 con 0x7fa6840d01f0 2023-03-24T22:24:56.516 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4035454282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa684160e90 con 0x7fa6840d01f0 2023-03-24T22:24:56.516 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa67a7fc700 1 -- 172.21.15.18:0/4035454282 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674003a60 con 0x7fa6840d01f0 2023-03-24T22:24:56.516 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.508+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4035454282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa684160bb0 con 0x7fa6840d01f0 2023-03-24T22:24:56.516 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.509+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4035454282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa684161170 con 0x7fa6840d01f0 2023-03-24T22:24:56.516 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.510+0000 7fa67a7fc700 1 -- 172.21.15.18:0/4035454282 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fa674003bc0 con 0x7fa6840d01f0 2023-03-24T22:24:56.516 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.510+0000 7fa67a7fc700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fa668069de0 0x7fa66806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-24T22:24:56.516 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.510+0000 7fa67a7fc700 1 -- 172.21.15.18:0/4035454282 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fa674021070 con 0x7fa6840d01f0 2023-03-24T22:24:56.517 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.511+0000 7fa689b9a700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fa668069de0 0x7fa66806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-24T22:24:56.517 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.511+0000 7fa689b9a700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fa668069de0 0x7fa66806c290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa6800011e0 tx=0x7fa680009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-24T22:24:56.517 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.515+0000 7fa67a7fc700 1 -- 172.21.15.18:0/4035454282 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fa6740551c0 con 0x7fa6840d01f0 2023-03-24T22:24:56.712 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:56.711+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4035454282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa68404e850 con 0x7fa6840d01f0 2023-03-24T22:24:57.352 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:57.351+0000 7fa67a7fc700 1 -- 172.21.15.18:0/4035454282 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa6740587e0 con 0x7fa6840d01f0 2023-03-24T22:24:57.356 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:57.355+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fa668069de0 msgr2=0x7fa66806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:57.356 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:57.355+0000 7fa68bdfe700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fa668069de0 0x7fa66806c290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa6800011e0 tx=0x7fa680009040 comp rx=0 tx=0).stop 2023-03-24T22:24:57.356 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:57.355+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4035454282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6840d01f0 msgr2=0x7fa684159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-24T22:24:57.356 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:57.355+0000 7fa68bdfe700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6840d01f0 0x7fa684159f10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa674000c00 tx=0x7fa674004750 comp rx=0 tx=0).stop 2023-03-24T22:24:57.356 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:57.355+0000 7fa68bdfe700 1 -- 172.21.15.18:0/4035454282 shutdown_connections 2023-03-24T22:24:57.356 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:57.355+0000 7fa68bdfe700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:6832/114514,v1:172.21.15.100:6833/114514] conn(0x7fa668069de0 0x7fa66806c290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:57.357 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:57.355+0000 7fa68bdfe700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68415c9b0 0x7fa6840c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22:24:57.357 INFO:teuthology.orchestra.run.smithi018.stderr:2023-03-24T22:24:57.355+0000 7fa68bdfe700 1 --2- 172.21.15.18:0/4035454282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa684070f90 0x7fa6841e7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-24T22: